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The  following  abbreviations  and  acronyms  are  used  in  this  report. 

AMRAAM . Advanced  Medium  Range  Air-to-Air  Missile 

AFOTEC . Air  Force  Operational  Test  and  Evaluation  Center 

Army  Board . U.S.  Army  Air  Defense  Artillery  Board 

Army  School . U.S.  Army  Air  Defense  Artillery  School 

BDM . BDM  International  Inc. 

Booz-Allen . Booz-Allen  &  Hamilton  Incorporated 

CAAS . Contracted  Advisory  and  Assistance  Services 

CAS . CAS  Incorporated 

Colsa . . . Colsa  Incorporated 

Development . Development,  Production,  or  Testing 

Director/DOT&E . Director  of  Operational  Test  and  Evaluation 

FAR . Federal  Acquisition  Regulation 

CAO . General  Accounting  Office 

IDA . Institute  for  Defense  Analysis 

MACA . . Management  Assistance  Corporation  of  America 

McLaughlin . McLaughlin  Research  Corporation 

MCOTEA . Marine  Corps  Operational  Test  and  Evaluation  Activity 

MILSTAR . Military,  Strategic,  Tactical  and  Relay  System 

OMB . . Office  of  Management  and  Budget 

Operational  Test . Operational  Test  and  Evaluation 

OPTEC . Army  Operational  Test  and  Evaluation  Command 

OPTEVFOR . Navy  Operational  Test  and  Evaluation  Force 

OTEA . . Army  Operational  Test  and  Evaluation  Agency 

Radar . Over-The-Horizon  Backscatter  Radar  System 

SAIC . Science  Applications  International  Corporation 

Services . . . . Advisory  and  Assistance  Services 

Test  Agency . Operational  Test  Agency 

Test  Center . Pacific  Missile  Test  Center 

Veda . . Veda  Incorporated 
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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION 
ASSISTANT  SECRETARY  OF  DEFENSE  (FORCE 
MANAGEMENT  AND  PERSONNEL) 

DIRECTOR  OF  DEFENSE  PROCUREMENT 
ASSISTANT  SECRETARY  OF  THE  ARMY  (FINANCIAL 
MANAGEMENT) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (FINANCIAL 
MANAGEMENT) 

ASSISTANT  SECRETARY  OF  THE  AIR  FORCE  (FINANCIAL 
MANAGEMENT  AND  COMPTROLLER) 

DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 

SUBJECT:  Audit  Report  on  Consulting  Services  Contracts  for 

Operational  Test  and  Evaluation  (Report  No. 


This  final  report  is  provided  for  your  information  and  use. 
The  audit  was  made  in  response  to  a  request  by  Representative 
Barbara  Boxer  and  addresses  the  advisory  and  assistance  services 
contractors  that  participated  in  the  development,  production, 
testing  or  operational  test  and  evaluation  of  major  Defense 
systems.  Management  comments  were  considered  in  preparing  this 
report . 

The  DoD  Directive  7650.3  requires  all  audit  recommendations 
to  be  resolved  promptly.  Therefore,  all  addressees  except  for 
the  Under  Secretary  of  Defense  for  Acquisition  and  the  Director 
of  Defense  Procurement  must  provide  final  comments  on  the 
unresolved  recommendations  and  monetary  benefits  by  October  21, 
1991.  See  the  "Status  of  Recommendations"  section  at  the  end  of 
each  finding  for  the  unresolved  recommendations  and  the  specific 
requirements  for  your  comments. 

The  DoD  Directive  7650.3  also  requires  that  comments 
indicate  concurrence  or  nonconcurrence  in  the  finding  and  each 
recommendation  addressed  to  you.  If  you  concur,  describe  the 
corrective  actions  taken  or  planned,  the  completion  dates  for 
actions  already  taken,  and  the  estimated  dates  for  completion  of 
planned  actions.  If  you  nonconcur,  you  must  state  your  specific 
reasons  for  each  nonconcurrence.  If  appropriate,  you  may  propose 
alternative  methods  for  accomplishing  desired  improvements. 
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If  you  nonconcur  with  the  estimated  monetary  benefits  or  any 
part  thereof,  you  must  state  the  amounts  you  nonconcur  with  and 
the  basis  for  your  nonconcurrence.  Recommendations  and  potential 
monetary  benefits  are  subject  to  resolution  in  accordance  with 
DoD  Directive  7650.3  in  the  event  of  nonconcurrence  or  failure  to 
comment.  We  also  ask  that  your  comments  indicate  concurrence  or 
nonconcurrence  with  the  internal  control  weakness  highlighted  in 
Part  I. 


The  courtesies  extended  to  the  staff  are  appreciated.  If 
you  have  any  questions  on  this  audit,  please  contact  Mr.  Garold 
E.  Stephenson,  Program  Director,  at  (703)  614-6275  (DSN  224-6275) 
or  Mr.  Henry  F.  Kleinknecht,  Project  Manager,  at  (703)  614-3461 
(DSN  224-3461).  The  planned  distribution  of  this  report  is 
listed  in  Appendix  T. 


Robert  J.  Lieberman 
Assistant  Inspector  General 
for  Auditing 


Enclosure 


cc: 

Secretary  of  the  Army 
Secretary  of  the  Navy 
Secretary  of  the  Air  Force 

Director,  Defense  Acquisition  Regulations  System 


Office  of  the  Inspector  General,  DoD 


AUDIT  REPORT  NO.  91-115  August  22,  1991 

(Project  No.  OCH-5009) 

CONSULTING  SERVICES  CONTRACTS  FOR  OPERATIONAL  TEST  AND  EVALUATION 

EXECUTIVE  SUMMARY 


Introduction.  The  Director  of  Operational  Test  and  Evaluation 
(the  Director)  is  the  principal  advisor  to  the  Secretary  of 
Defense  and  the  Under  Secretary  of  Defense  for  Acquisition  on 
Operational  Test  and  Evaluation  (operational  tests)  in  DoD.  Each 
DoD  Component  has  one  major  field  Operational  Test  Agency  that  is 
responsible  for  planning,  conducting,  and  reporting  all 
operational  tests  for  major  Defense  acquisition  programs. 

Objectives.  The  overall  objective  was  to  determine  whether  the 
same  advisory  and  assistance  services  (services)  contractors  that 
participated  in  the  development,  production,  or  testing 
(development)  of  major  Defense  acquisition  systems  were  also 
involved  in  the  operational  tests  of  those  systems.  In  addition, 
we  determined  whether  using  services  contracts  to  provide  support 
for  operational  tests  was  more  cost-effective  than  developing  a 
capability  to  perform  the  work  in-house.  We  also  evaluated  the 
effectiveness  of  applicable  internal  controls. 

Audit  Results.  The  Military  Departments'  Operational  Test 
Agencies  (Test  agencies)  frequently  used  the  same  services 
contractors  to  support  operational  tests  for  major  Defense 
acquisition  systems  that  participated  in  the  development  of  the 
systems.  As  a  result,  operational  tests  did  not  attain  the 
desired  impartiality  and  independence,  test  assessments  may  be 
biased,  and  systems  may  be  produced  and  deployed  with  unknown 
performance  limitations  (Finding  A). 

The  Director  and  the  Test  agencies  also  used  repeated  and 
extended  services  contracts  to  support  operational  tests  that 
were  not  as  cost-effective  as  developing  an  in-house  capability 
to  perform  the  work.  As  a  result,  DoD  was  dependent  on  the 
services  contractors  for  program  continuity  and  the  "corporate 
knowledge"  needed  to  plan,  analyze,  and  report  operational  tests, 
and  spent  over  $44  million  annually  for  contractor  assistance 
that  was  not  as  cost-effective  as  developing  an  in-house 
capability  (Finding  B) . 

Internal  Controls.  Internal  controls  were  not  adequate  to 
prevent  the  services  contractors  that  had  participated  in 
development  of  major  Defense  acquisitions  systems  from  supporting 
operational  tests  for  the  systems  (Finding  A).  See  internal 
controls  section  in  Part  I  of  this  report  for  more  details. 
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Potential  Benefits  of  Audit.  The  audit  showed  that  the  Director 
and  the  Test  agencies  could  reduce  costs  by  about  $26.1  million 
for  FY's  1992  through  1996  by  decreasing  their  services  contracts 
60  percent  and  developing  an  in-house  capability  to  support 
operational  tests  (Appendix  R). 

Summary  of  Recommendations.  We  recommended  additional 
procedures,  legislative  changes,  internal  controls,  and  replacing 
services  contractors  with  in-house  civilian  employees. 

Management  Comments.  The  Director  concurred  with  Finding  A 
recommendations  to  require  program  managers  to  identify  services 
contractors  used  for  development  and  to  develop  a  standard 
organizational  conflict  of  interest  clause.  The  Director  did  not 
believe  there  was  a  need  for  legislation  that  would  allow  Test 
agencies  to  obtain  waivers  to  use  the  same  services  contractors 
to  support  operational  tests  that  participated  in  development. 
The  Director  of  Defense  Procurement  agreed  to  include  the 
conflict  of  interest  clause  in  DoD  regulations. 

The  Army,  Navy,  and  Air  Force  concurred  with  Finding  A 
recommendations  to  insert  conflict  of  interest  clauses  in 
services  contracts  and  to  direct  contracting  officers  to  enforce 
the  provisions.  The  Army  did  not  believe  any  conflicts  of 
interest  had  occurred,  the  Navy  agreed  to  take  action  to  remove 
existing  conflicts  of  interest,  and  the  Air  Force  thought  that 
the  same  contractors  should  be  able  to  support  operational  tests 
and  development  when  "adequate  safeguards"  were  in  place. 

The  Director,  the  Assistant  Secretary  of  Defense  (Force 
Management  and  Personnel),  and  the  Navy  basically  nonconcured 
with  Finding  B  recommendations  to  hire  additional  civilian 
personnel  and  reduce  their  reliance  on  services  contractors.  The 
Army  concurred  with  the  recommendations  and  the  Air  Force 
basically  concurred  pending  an  analysis  of  the  costs  and 
benefits.  - 

The  DoD  Director  of  Contracted  Advisory  and  Assistance  Services 
(CAAS),  commented  that  its  current  CAAS  definition  excluded  the 
contractor  services  used  by  the  Test  agencies. 

We  request  that  the  Director  of  Operational  Test  and  Evaluation, 
the  Assistant  Secretary  of  Defense  (Force  Management  and 
Personnel),  the  Army  Operational  Test  and  Evaluation  Command,  the 
Navy  Operational  Test  and  Evaluation  Force,  the  Air  Force 
Operational  Test  and  Evaluation  Center,  and  the  Director  of 
Contracted  Advisory  and  Assistance  Services  provide  additional 
comments  to  the  final  report  by  October  21,  1991.  The  full 
discussion  of  the  responsiveness  of  management  comments  is 
included  in  Part  II  of  the  report,  and  the  complete  text  of 
management  comments  is  included  in  Part  IV  of  the  report. 
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PART  I  -  INTRODUCTION 


Background 

Operational  Test  and  Evaluation  (operational  test)  is  the  field 
test,  under  realistic  conditions,  of  any  item  (or  key  component) 
of  weapons,  equipment,  or  munitions  for  determining  their 
effectiveness  and  suitability  in  combat  by  typical  military  users 
and  the  evaluation  of  the  results  of  such  tests.  The  primary 
purpose  of  operational  tests  is  to  ensure  that  only  operationally 
effective  and  suitable  systems  are  delivered  to  the  operating 
forces. 

The  Director  of  Operational  Test  and  Evaluation  (the  Director)  is 
the  principal  advisor  to  the  Secretary  of  Defense  and  the  Under 
Secretary  of  Defense  for  Acquisition  within  DoD.  The  Director 
prescribes  policies  and  procedures  for  the  conduct  of  operational 
tests,  monitors  and  reviews  operational  tests,  and  coordinates 
operational  tests  conducted  jointly  by  more  than  one  Military 
Department  or  Defense  agency. 

In  each  DoD  Component,  one  major  field  Operational  Test  Agency 
(Test  agency)  is  responsible  for  planning  and  conducting  all 
operational  tests.  In  addition,  the  Test  agency  conducts 
operational  tests,  reports  test  results,  and  provides  an 
evaluation  of  the  tested  system's  operational  effectiveness  and 
suitability  directly  to  the  appropriate  Service  chief  or  Defense 
agency  director.  The  Test  agency  is  independent  from  the 
material  developing/procuring  agency  and  the  using  agency. 

The  Director  oversees  operational  tests  conducted  by  the 
four  Test  agencies  within  the  Services,  which  include:  the  Army 
Operational  Test  and  Evaluation  Agency  (OTEA),  the  Navy 
Operational  Test  and  Evaluation  Force  (OPTEVFOR),  the  Air  Force 
Operational  Test  and  Evaluation  Center  (AFOTEC),  and  the  Marine 
Corps  Operational  Test  and  Evaluation  Activity  (MCOTEA).  In 
November  1990,  the  Army's  Operational  Test  Agency  was  reorganized 
into  the  Army  Operational  Test  and  Evaluation  Command  (OPTEC). 

Objectives 

The  overall  objective  of  this  audit  was  to  determine  whether  the 
same  services  contractors  that  participated  in  the  development  of 
major  Defense  acquisition  systems  were  also  involved  in  the 
operational  tests  of  those  systems.  We  also  evaluated  the 
effectiveness  of  applicable  internal  controls.  We  expanded  the 
scope  of  our  objective  during  the  survey  phase  to  determined 
whether  using  services  contracts  was  more  cost-effective  than 
developing  a  capability  to  perform  the  work  in-house. 


Scope 


Annual  report  on  operational  tests.  We  initially  selected 
four  systems  from  each  Military  Department  from  the  Director's 
FY  1989  Annual  Report  on  Operational  Tests.  We  did  not  include 
the  Marine  Corps  in  our  review  because  no  programs  were  described 
in  the  report.  For  the  Navy,  we  changed  our  approach  and 
reviewed  two  systems  and  each  of  the  Test  agency's  services 
contracts  (excluding  classified  contracts)  and  determined  whether 
any  of  the  contractors  or  subcontractors  were  also  supporting 
development . 

Services  contracts  used  in  FY's  1989  and  1990.  We  visited 
the  program  management  office,  development  or  technical  test 
sites,  and  the  operational  test  sites  to  review  the  services 
contracts  used  in  FY's  1989  and  1990.  We  determined  the  costs  of 
these  contracts,  the  contractors  and  subcontractors  performing 
the  work,  and  the  specific  taskings  each  contractor  performed. 
We  also  determined  the  services  contractors  from  earlier  fiscal 
years. 

Hourly  costs  for  services  contractors.  We  analyzed  costs  on 
21  services  contracts  and  determined  the  hourly  costs  for  each  of 
the  contractor  personnel  categories.  We  compared  these  hourly 
costs  to  the  hourly  costs  of  comparable  civilian  Government 
employees.  The  Director  and  the  Test  agencies  spent  over 
$44  million  for  services  contractors  in  FY  1990. 

Audit  period,  reason  for  audit,  standards,  and  locations. 
This  program  audit  was  made  from  April  1990  through  January  1991 
at  the  request  of  Representative  Barbara  Boxer  who  was  interested 
in  knowing  to  what  extent  contractors  involved  in  the  development 
or  production  of  a  weapon  system  were  also  hired  to  consult  on 
the  operational  tests  of  those  systems,  and  for  what  purpose. 
The  audit  was  made  in  accordance  with  auditing  standards  issued 
by  the  Comptroller  General  of  the  United  States,  as  implemented 
by  the  Inspector  General,  DoD,  and  accordingly  included  such 
tests  of  internal  controls  as  were  considered  necessary.  The 
implementation  of  the  Federal  Managers'  Financial  Integrity  Act 
by  Defense  activities  to  strengthen  internal  controls  did  not 
specifically  relate  to  our  audit  objectives;  therefore,  an 
evaluation  was  not  possible  within  the  scope  of  our  audit.  We 
did  not  rely  on  any  computer-based  data  to  accomplish  the  audit 
objectives.  Activities  visited  or  contacted  are  listed  in 
Appendix  S. 

Internal  Controls 


The  audit  identified  material  internal  control  weaknesses  as 
defined  by  Public  Law  97-225,  Office  of  Management  and  Budget 
Circular  A-123,  and  DoD  Directive  5010.38.  Controls  were  not 
established  or  effective  to  prevent  services  contractors  from 
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supporting  operational  tests  and  development  for  major  Defense 
acquisition  systems.  All  recommendations  to  Finding  A  in  this 
report,  if  implemented,  will  correct  the  weaknesses.  We  have 
determined  that  monetary  benefits  will  not  be  realized  by 
implementing  recommendations  to  Finding  A.  A  copy  of  this  report 
will  be  provided  to  the  senior  official  responsible  for  internal 
controls  within  the  Secretaries  of  the  Army,  Navy,  and  Air  Force. 

Prior  Audits  and  Other  Reviews 


General  Accounting  Office  (GAO)  reports. 

GAO  Report  No.  NSIAD-90-119 ,  (OSD  Case  8026-A), 
"CONSULTING  SERVICES:  Role  and  Use  in  Acquiring  Three  Weapon 
Systems,"  August  20,  1990,  stated  that  GAO  reviewed  three  DoD 
weapon  systems  to  determine: 

o  how  DoD  used  consulting  services  in  acquiring 

these  systems; 

o  how  the  systems'  contractors  used  consultants; 

o  whether  consultants  worked  for  both  the 
Government  and  Defense  contractors  on  these  systems,  and  if  so, 
whether  any  conflicts  of  interest  existed;  and 

o  how  well  DoD  identified  and  reported  its  use  of 
consulting  services. 

GAO  found  no  basis  to  conclude  that  conflicts  of  interest  existed 
in  the  three  instances  it  identified  where  consultants  worked  for 
both  the  Government  and  a  Defense  contractor  on  matters  related 
to  the  same  weapon  system.  GAO's  review,  however,  did  highlight 
key  principles  to  guide  the  Government's  approach  to  addressing 
conflicts  of  interest  concerns,  among  them  the  need  for 
Government  awareness  of  consultant  employment  relationships  in 
order  to  make  informed  judgment  about  potential  conflicts  and  the 
use  of  appropriate  contract  clauses  to  avoid  or  mitigate 
identified  conflicts.  No  recommendations  were  made  in  this  area. 

GAO  also  found  that  DoD  did  not  accurately  identify  or  report  its 
use  of  consulting  services,  due  to  difficulties  in  interpreting 
the  definitions  of  these  services  or  other  internal  control 
weaknesses.  For  this  finding,  GAO  recommended  that  the  Secretary 
of  Defense: 

o  review  and  clarify  existing  guidance  on  CAAS  to  preclude 
differing  interpretations  among  the  Services; 
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o  direct  the  DoD  and  component  CAAS  directors  to  strengthen 
their  review  procedures  to  ensure  that  the  Services  accurately 
report  CAAS  budget  data;  and 

o  direct  the  Secretaries  of  the  Army,  Navy,  and  Air  Force 
to  review,  and  where  necessary,  develop  or  revise  their 
individual  instructions  and  procedures  to  ensure  that  CAAS  is 
accurately  identified  and  entered  into  the  accounting  system. 

DoD  concurred  with  the  finding  and  recommendations  and  stated 
that  it  will  develop  an  "action  plan"  to  strengthen  CAAS 
management  and  reporting  controls. 

GAO  Report  No.  NSIAD-91-60,  (OSD  Case  8382),  "TEST  AND 
EVALUATION:  The  Director,  Operational  Test  and  Evaluation's 

Controls  Over  Contractors,"  December  21,  1990,  addressed  the 
Director's  management  of  its  contracts,  controls  over  conflicts 
of  interest,  the  appropriateness  of  the  use  of  the  Institute  for 
Defense  Analysis,  and  the  Institute's  conflict  of  interest 
controls  over  its  use  of  consultants  and  subcontractors.  GAO 
found  that  the  Director  was  complying  with  existing  policies  and 
procedures  relating  to  possible  conflicts  of  interest  but  was 
concerned  about  work  performed  by  the  Institute.  GAO  made  no 
recommendations . 

GAO  Draft  Report  Code  966408,  (OSD  Case  8772), 
"GOVERNMENT  CONSULTANTS:  Are  Contract  Consultants  Performing 
Inherently  Governmental  Functions,"  July  19,  1991,  addressed  the 
issue  of  determining  what  functions  are  inherently  Governmental. 
GAO  found  that  it  was  difficult  to  apply  the  Office  of  Management 
and  Budget  (OMB's)  broad  definition  of  inherently  Governmental 
functions  objectively  to  functions  not  specifically  listed  in  the 
guidance.  GAO  also  found  that  regardless  of  whether  the 
Government  carries  out  activities  with  its  own  employees  or  by 
contract,  it  must  have  the  core  capability  and  a  sufficient 
number  of  trained  and  experienced  staff  to  properly  manage  and  be 
accountable  for  its  work.  GAO  recommended  that  OMB: 

o  clarify  guidance  to  agencies  on  contracting  for 
consulting  services, 

o  compile  a  short  generic  list  of  functions  which 
should  never  be  contracted  out,  and 

o  require  implementing  instructions  from  each 

agency. 

GAO  also  recommended  that  the  Senate  Committee  on  Governmental 
Affairs : 

o  hold  hearings  on  the  revised  guidance  to  ensure  that  it 
was  consistent  with  congressional  views  on  the  subject,  and 
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o  provide  agencies  with  the  authority  and  flexibility  to 
use  Government  employees  for  activities  that  may  be  considered 
inherently  Governmental  in  nature. 

DoD  Office  of  the  Assistant  Inspector  General-Inspections. 
Inspection  Report  No.  91-INS-09,  "Operational  Test  and  Evaluation 
Within  the  Department  of  Defense,"  May  24,  1991.  The  inspection 
report  found  that: 

o  the  present  operational  test  structure  was  dependent 
on  the  acquisition  system  and  should  have  more  independence; 

o  operational  test  processes  were  well  designed 
conceptually,  but  did  not  always  function  optimally  because  of 
problems  in  the  operational  test  structure;  and 

o  DoD  had  not  developed  systems  for  selection, 
assignment,  and  continuous  training  of  operational  test 
personnel . 

Recommendations  were  made  for  changes  in  legislation, 
organization,  and  policy  and  procedures  designed  to  improve 
operational  tests  by  addressing  the  systemic  problems  identified. 
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PART  II  -  FINDINGS  AND  RECOMMENDATIONS 


A.  SERVICES  CONTRACTORS  SUPPORTING  OPERATIONAL  TESTS  AND 
DEVELOPMENT 


The  Test  agencies  frequently  used  the  same  services  contractors 
to  support  the  operational  tests  for  major  Defense  acquisition 
systems  that  participated  in  the  development  of  the  systems 
because  the  internal  controls  were  not  adequate.  In  addition, 
the  services  contracts  did  not  always  include  conflicts  of 
interest  clauses  that  prohibited  services  contractors  from 
supporting  the  operational  tests  and  the  development. 
Furthermore,  when  the  contracts  did  contain  the  proper  clauses, 
the  contractors  and  subcontractors  did  not  always  adhere  to  the 
clauses.  As  a  result,  operational  tests  did  not  attain  the 
required  impartiality  and  independence,  test  assessments  may  have 
been  biased,  and  systems  may  have  been  produced  and  deployed  with 
unknown  performance  limitations. 

DISCUSSION  OF  DETAILS 


Background 

10  U.S.C.  2399,  as  implemented  by  Public  Law  101-189,  November 
29,  1989,  "Operational  Test  and  Evaluation  of  Defense  Acquisition 
Programs,"  subsection  (e)  "Impartial  Contracted  Advisory  and 
Assistance  Services,"  part  (3),  states  that: 

A  contractor  that  has  participated  in  (or  is 
participating  in)  the  development,  production,  or 
testing  of  a  system  for  a  Military  Department  or 
Defense  Agency  (or  for  another  contractor  of  the 
Department  of  Defense)  may  not  be  involved  (in  any 
way)  in  the  establishment  of  criteria  for  data 
collection,  performance  assessment,  or  evaluation 
activities  for  the  operational  test  and  evaluation. 

Army  Systems 

The  same  services  contractors  supported  the  development  and  the 
operational  tests  for  the  Pedestal  Mounted  Stinger  system  and  the 
Line  of  Sight-Forward-Heavy  system.  There  were  no  conflicts  of 
interest  relating  to  the  M109A3E2  Howitzer  Improvement  Program  or 
the  Joint  Surveillance  Target  Attack  Radar  System,  Ground  Station 
Module.  Details  of  the  two  systems  with  conflicts  of  interest 
follow. 

Pedestal  Mounted  Stinger  System.  The  Project  Office  at  the 
Army  Missile  Command  is  responsible  for  life-cycle  management  of 
the  system  including  development,  technical  test  and  evaluation 
acquisition,  procurement,  and  production. 
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The  U.S.  Army  Air  Defense  Artillery  School  (the  Army  School)  at 
Fort  Bliss,  Texas,  conducts  a  wide  range  of  test  and  evaluation, 
and  combat  development  activities  for  Army  air  defense  artillery 
systems,  including  the  Pedestal  Mounted  Stinger  system.  The  Army 
School  was  the  independent  evaluator  for  force  development  test 
and  experimentation  phases  I  and  II  for  the  system.  The  Army 
Operational  Test  Agency  was  the  initial  operational  test 
evaluator  for  the  system. 

Two  contractors,  Colsa,  Incorporated  (Colsa)  and  BDM 
International,  Inc.  (BDM)  participated  in  both  the  development 
and  the  operational  tests  of  the  system  (Appendix  A).  The  tasks 
performed  by  the  two  contractors  follow. 

Development.  The  Project  Office  had  a  services  contract 
that  was  awarded  to  Colsa  in  November  1987  to  provide  system 
engineering  and  technical  assistance  during  the  development  phase 
of  the  system.  The  contractor  was  tasked  to  attend  program 
meetings;  analyze  presentations  and  discussions;  provide 
recommendations;  develop  data;  and  perform  independent  analyses 
for  management,  technical,  and  evaluation  support  to  the  Program 
Manager  for  the  system  and  all  system  derivatives.  As  of 
September  1990,  the  total  contract  price  was  about  §9  million. 

The  Project  Office  contracted  with  BDM  to  develop  a  Performance 
Analysis  Database  and  support  concept  definition/formulation  to 
support  out-year  planning  for  the  system.  The  work  was  done  in 
direct  support  of  the  Army  Missile  Command's  Research, 
Development,  and  Engineering  Center  and  was  funded  by  the  Project 
Engineer. 

Operational  tests.  The  Concepts  and  Studies  Division  of 
the  Army  School  had  a  services  contract  that  was  awarded  to  Colsa 
in  September  1987  to  provide  general  technical  and  analytical 
services  support.  The  contractor  was  required  to  develop  air 
defense  artillery  technological  concepts  and  simulation/analysis 
of  system  performance,  tactics,  and  doctrine  associated  with 
current,  new,  or  improved  air  defense  artillery  systems. 
Contract  efforts  included  contributing  to  the  preparation  of 
organizational  and  operational  plans,  and  test  issues  and 
criteria  documents.  As  of  September  1990,  the  contract  was 
funded  for  about  $5.7  million. 

The  Materiel  and  Logistics  Systems  Division  of  the  Army  School, 
Directorate  of  Combat  Developments,  had  a  second  services 
contract  that  was  awarded  to  Colsa  in  January  1988  to  provide 
technical  support  to  review  and  analyze  forward  area  air  defense 
systems  requirements  and  test  results.  As  of  September  1990, 
this  contract  was  funded  for  about  $6.8  million. 

The  Test  and  Evaluation  Division  of  the  Army  School,  Directorate 
of  Combat  Developments,  had  a  third  services  contract  that  was 
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awarded  to  Colsa  in  May  1988  to  perform  and  document  studies  and 
analyses  in  support  of  Army  materiel  systems'  test  and 
evaluation.  Specific  tasks  required  the  contractor  to  prepare 
the  following  items  for  the  system:  the  Critical  Issues  and 
Criteria  document,  the  Independent  Evaluation  Plan  for  force 
development  test  and  experimentation  phase  II,  and  the 
Independent  Evaluation  Report  for  force  development  test  and 
experimentation  phase  I.  This  contract  was  funded  for  about 
$1.2  million. 

The  Colsa  contracts  did  not  include  organizational  conflict  of 
interest  clauses  that  addressed  the  issue  of  contractors 
supporting  Operational  Tests  and  Development. 

The  Army  Operational  Test  Agency  had  a  $25  million  services 
contract  that  was  awarded  to  BDM  in  July  1988  for  a  5-year 
period.  In  January  1989,  the  Army  tasked  BDM  to  provide 
technical  services  in  support  of  continuous  and  comprehensive 
evaluation  of  the  system,  at  a  cost  of  about  $1.1  million.  The 
technical  support  included  reviewing  test  documentation, 
performing  data  analysis  in  support  of  operational  assessments 
and  the  independent  evaluation  report,  and  providing  technical 
assistance  in  preparation  of  evaluation  briefings  and  reports. 
BDM  used  the  Management  Assistance  Corporation  of  America  (MACA) 
as  its  major  subcontractor. 

The  contract  included  an  organizational  conflict  of  interest 
clause  that  required  the  contractor  and  its  subcontractors  to 
notify  the  contracting  officer  of  any  contracts  or  subcontracts 
to  support  the  cognizant  DoD  program  or  project  manager.  In 
January  1991,  BDM  notified  the  Army  Operational  Test  Agency  and 
the  contracting  officer  of  a  potential  conflict  of  interest 
relating  to  the  support  that  BDM  provided  to  the  Army  Missile 
Command  for  the  Project  Office. 

Line  of  Sight-Forward-Heavy  System.  The  Project  Management 
Office  for  the  system,  which  is  collocated  and  supported  by  the 
Army  Missile  Command,  is  responsible  for  life-cycle  management  of 
the  system  including  development,  technical  test  and  evaluation, 
acquisition,  procurement,  and  production. 

The  U.S.  Army  Air  Defense  Artillery  Board  (the  Army  Soar'd)  is  an 
element  of  the  Test  and  Experimentation  Command  whose  primary 
functions  are  operational  testing,  evaluation,  analyses,  and 
combat  development.  The  Army  Board  was  the  force  development 
test  and  experimentation  phase  I  tester  for  the  system,  and  the 
Army  Operational  Test  Agency  was  the  initial  operational  test 
evaluator  for  the  system. 

One  contractor,  CAS,  Incorporated  (CAS)  participated  in  the 
operational  tests  and  the  development  of  the  system 
(Appendix  B) .  Details  of  the  tasks  CAS  performed  are  as  follows. 


9 


Development.  In  February  1988,  the  Project  Management 
Office  funded  a  task  order  for  CAS  to  provide  technical  support 
at  a  cost  of  about  $1.1  million. 

The  Project  Management  Office  had  a  follow-on  contract  that  was 
awarded  to  Native  American  Services  Associates  in  December  1989 
to  provide  technical  support  in  the  areas  of  system  integration, 
logistics,  production  engineering,  and  program  management  in 
support  of  the  continuing  system  acquisition  process.  The 
contract  had  a  value  of  about  $1.3  million,  and  CAS  was  the  major 
subcontractor . 

The  Army  Materiel  Test  and  Evaluation  Directorate  had  a  services 
contract  with  United  International  Engineering.  In  April  1990, 
the  Directorate  tasked  the  contractor  to  provide  system  analysis 
support  for  Technical  Test  Phases  A,  B,  and  D  and  also  for 
Operational  Tests  of  missile  firing  missions.  The  estimated  cost 
of  the  task  was  about  $228,000.  Again  CAS  was  a  major 
subcontractor  providing  support. 

Operational  tests.  The  Army  Board  had  a  services 
contract  that  was  awarded  to  MACA  in  March  1987  for  support  in 
performing  its  mission.  General  tasks  to  be  performed  included 
support  services  for  analysis,  testing,  integrated  logistics, 
engineering,  operations,  management,  and  for  other  areas.  From 
March  1987  through  September  1990,  the  contract  was  funded  for 
about  $12.6  million,  and  CAS  was  MACA's  major  subcontractor. 
Examples  of  the  tasks  CAS  performed  relating  to  the  system 
included  preparing  Test  Planning  reports,  Operational  Test 
Readiness  Reviews,  Force  Development  Test  and  Experimentation 
Phase  II  results  reports,  and  Early  Operational  Assessments. 

The  Army  Operational  Test  Agency  tasked  BDM  to  provide  support 
for  operational  tests  of  the  system  under  its  services 
contract.  Four  delivery  orders  totaling  about  $2.3  million  were 
issued  to  support  the  Army  Operational  Test  Agency  with  its 
mission  to  evaluate  the  system.  BDM  used  CAS  and  MACA  as  its 
major  subcontractors  to  perform  the  tasks  that  included: 
oversight,  data  analysis,  technical  support  and  status  reporting 
for  technical  testing;  development  of  a  data  reduction  scheme  and 
an  implementation  plan  for  end-to-end  data  retrieval  for  the 
systems  initial  operational  test;  and  technical  services  in 
support  of  the  Army  Operational  Test  Agency's  participation  in 
force  development  test  and  experimentation  phase  II  and  the 
initial  operational  tests  of  the  system. 

The  Army's  contract  with  BDM  contained  an  organizational  conflict 
of  interest  clause  that  required  the  contractor  and  its 
subcontractors  to  notify  the  contracting  officer  of  any  contracts 
or  subcontracts  to  support  the  cognizant  DoD  program  or  Project 
Manager.  However,  neither  BDM  nor  CAS  informed  the  contracting 
officer  that  CAS  had  supported  the  Project  Management  Office  and 
the  Army  Materiel  Test  and  Evaluation  Directorate  with  the 
development  of  the  system. 
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Navy  Systems 

The  same  services  contractor  supported  the  development  and  the 
operational  tests  for  the  P-14  TOMCAT  and  Advanced  Medium  Range 
Air-to-Air  Missile  (AMRAAM)  systems.  There  were  no  conflicts  of 
interest  on  the  other  five  services  contracts  used  by  the  Navy 
Operational  Test  Agency  and  reviewed  as  a  part  of  this  audit. 
Details  of  the  organizational  conflicts  of  interest  relating  to 
the  F-14  TOMCAT  system  follow,  and  the  AMRAAM  system  is  discussed 
under  Air  Force  systems. 

F-14  TOMCAT  System.  The  Pacific  Missile  Test  Center  (Test 
Center)  performs  development,  test,  evaluation,  and  follow-on 
engineering,  logistics,  and  training  support  for  Naval  weapons, 
weapon  systems,  and  related  devices.  The  Naval  Air  Development 
Center  performs  warfare  mission  analyses,  engineering  analyses, 
engineering  development  for  aircraft  and  avionics  systems  and 
subsystems,  and  the  integration  of  air  combat  systems  into  Naval 
aircraft.  Both  Centers  are  under  the  cognizance  of  the  Naval  Air 
Systems  Command.  The  Navy  Operational  Test  Agency  has  a 
detachment  located  at  the  Test  Center  which  was  assigned 
responsibility  for  planning,  conducting,  and  reporting 
operational  tests  for  the  F-14  TOMCAT  system. 

One  contractor,  Veda,  Incorporated  (Veda)  participated  in  the 
operational  tests  and  the  development  of  the  F-14  TOMCAT  system 
(Appendix  C) .  In  addition,  the  same  Veda  employees  worked  on 
both  the  operational  tests  and  the  development  of  the  F-14  TOMCAT 
system.  Details  of  the  tasks  Veda  performed  are  as  follows. 

Development.  The  Flight  Test  Division  at  the  Test 
Center  had  a  Services  contract  with  Veda  to  provide  engineering 
and  technical  services  for  flight  test  planning,  analysis  of  test 
data,  preparation  of  test  and  problem  reports,  and  investigations 
for  system  problems.  The  task  order  contract  was  awarded  in 
September  1989,  with  a  base  year  and  4  option  years  at  a  total 
estimated  cost  of  $15.8  million.  Specific  systems  that  may  be 
supported  include  the  F-14,  A-6,  and  F-18  aircraft  and 
SIDEWINDER,  SPARROW,  PHOENIX,  and  AMRAAM  missiles.  The  Test 
Center  issued  Veda  two  task  orders  with  a  total  cost  not  to 
exceed  $1.2  million  to  provide  engineering  development,  and  test 
and  evaluation  support  for  the  F-14  TOMCAT  weapons  system  and 
tactical  software. 

The  Naval  Air  Development  Center  had  a  services  contract  with 
Veda  that  was  awarded  in  May  1988,  with  a  base  year  and  2  option 
years  at  a  total  estimated  cost  of  $9.6  million.  Specific 
systems  that  may  be  supported  included  the  F-14,  A-6,  F/A-18, 
AV-8B,  and  V-22  aircraft  and  the  AIM-54  Phoenix  and  AMRAAM 
missiles.  From  May  1988  through  May  1990,  about  $2.5  million  of 
the  work  performed  on  this  contract  supported  the  F-14  TOMCAT 
system. 
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Operational  tests.  The  Navy  Operational  Test  Agency’s 
detachment  at  the  Test  Center  had  a  services  contract  with  SIMSUM 
Operational  Research  to  provide  analytical  support  for  test 
planning,  project  operations,  and  test  analysis.  The  task  order 
contract  was  awarded  in  April  1986,  with  a  base  year  and  2  option 
years  at  a  total  estimated  cost  of  $2.3  million.  Several  tests 
to  be  supported  were  Follow-on  Operational  Test  and  Evaluation  of 
the  F/A-18A  aircraft  and  AIM-54C,  AIM-7M,  and  AIM-9M  missiles; 
and  Operational  Evaluation  of  the  F-14D  aircraft  and  the  AIM-120 
missile.  Veda  was  SIMSUM' s  major  subcontractor  on  the  contract. 

The  Operational  Test  Agency's  detachment  had  a  follow-on  services 
contract  with  Webster  Engineering  to  provide  analytical  support 
in  determining  the  operational  effectiveness  and  suitability  of 
weapons,  weapon  systems,  aircraft  and  aircraft  modifications,  and 
electronic  warfare  equipment.  The  task  order  contract  was 
awarded  in  August  1989,  with  a  base  year  and  4  option  years  at  a 
total  estimated  cost  of  $6.3  million.  Systems  that  may  require 
support  included  the  F/A-18A  and  F-14D  aircraft,  and  the  AIM-120, 
AIM-54C/C+ ,  AIM-7M  and  9M  missiles.  Veda  was  Webster 
Engineering's  major  subcontractor  on  this  contract.  The  Veda 
employees  who  supported  the  detachment  on  the  operational  tests 
for  the  F-14A  115A  tape,  the  test  planning  for  the  F-14D  IIC  Test 
Plan,  and  the  F-14D  and  AMRAAM  Operational  Evaluations  also 
supported  the  Flight  Test  Division  at  the  Test  Center  on  its  Veda 
contract  that  provided  development  support  for  the  F-14  TOMCAT 
system. 

The  Webster  Engineering  contract  contained  an  organizational 
conflict  of  interest  clause  that  required  the  contractor  to 
certify  that  he  and  his  affiliates  (subcontractors)  had  no 
interest  or  involvement  in  any  of  the  systems  that  the 
Operational  Test  Agency's  detachment  planned  to  operationally 
test  during  the  contract.  This  clause  should  have  precluded  Veda 
from  supporting  the  Navy  Operational  Test  Agency's  detachment. 
However,  Webster  Engineering  sent  the  contracting  officer  a 
letter  stating  that  neither  he  nor  Veda  felt  that  there  was  any 
conflict  of  interest  relevant  to  the  contract.  The  Webster 
Engineering  letter  was  based  on  a  Veda  letter  in  which  Veda 
stated  that  it  had  reviewed  the  organizational  conflicts  of 
interest  clause  and  that  it  was  in  compliance  with  the  clause  and 
would  remain  in  compliance. 

Air  Force  Systems 

The  same  services  contractors  supported  the  development  and  the 
operational  tests  for  the  AMRAAM;  Military,  Strategic,  Tactical 
and  Relay  (MILSTAR);  and  Over-The-Horizon  Backscatter  Radar 
(Radar)  systems.  There  was  no  conflict  of  interest  relating  to 
the  Low-Altitude  Navigation  and  Targeting  Inf rared-for-Night 
system.  Details  of  the  conflicts  of  interest  for  the  systems 
follow. 
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AMRAAM  System,  The  Joint  System  Program  Office,  collocated 
with  the  Air  Force  Systems  Command,  Munitions  Systems  Division, 
has  acquisition  management  responsibility  for  the  AMRAAM 
program.  Since  the  Navy  also  uses  the  AMRAAM  system,  the 
developmental  test  and  evaluation  was  partially  conducted  at  the 
Navy  Test  Center.  The  Navy's  operational  test  detachment  at  the 
Test  Center  was  assigned  responsibility  for  planning,  conducting, 
and  reporting  the  Navy's  operational  tests  for  the  AMRAAM  system. 
The  Air  Force  Operational  Test  Agency  was  responsible  for 
planning,  conducting,  and  reporting  the  Air  Force's  initial 
operational  tests  for  the  AMRAAM  system. 

Veda  and  the  McLaughlin  Research  Corporation  (McLaughlin)  had 
participated  in  the  development  and  in  the  operational  tests  of 
the  AMRAAM  system  (Appendix  D).  Details  of  the  tasks  Veda  and 
McLaughlin  performed  are  as  follows. 

Development.  The  program  office  had  a  services  contract 
with  Veda  that  was  awarded  in  June  1989  to  provide  technical 
support  for  the  management,  analysis,  and  interpretation  of  the 
AMRAAM  test  data  base.  Included  in  the  scope  were  operation  and 
maintenance  of  the  AMRAAM  data  base  management  system; 
preparation  of  technical  reports  and  documented  performance  of 
the  AMRAAM  system;  designing,  coding,  developing  and  testing  of 
enhancements  to  the  management  system;  and  assistance  in  the 
preparation  of  flight  test  data  analysis.  The  estimated  cost  of 
the  contract  was  $3  million 

In  January  1989,  the  Navy  Test  Center  contracted  with  McLaughlin 
to  provide  engineering  evaluation,  analysis,  and  test  support 
services  for  the  AMRAAM  system.  The  scope  included  analyzing  and 
evaluating  AMRAAM  missile  flight  test/simulation  data,  preparing 
system  level  documentation  and  flight  test  reports,  providing 
documentation  of  specific  AMRAAM  program  elements,  and 
documenting  Government  test  plans  for  AMRAAM  simulation  and 
flight  testing.  The  estimated  cost  of  the  contract  was 
$7.1  million. 

Operational  tests.  The  Navy  Operational  Test  Agency's 
detachment  at  the  Test  Center  contracted  with  Webster  Engineering 
to  provide  analytical  support  services  during  operational  tests 
of  the  AMRAAM  system.  Webster's  subcontractor,  Veda,  Inc., 
monitored  air/launch  and  AMRAAM  captive  equipment  missions, 
validated  and  analyzed  missile  data,  coordinated  with  Test  Center 
for  lethality  information,  and  supplied  the  effectiveness  portion 
of  the  final  operational  test  report.  We  also  determined  that 
several  Veda  employees,  who  supported  the  operational  tests,  also 
worked  on  a  Veda  contract  for  the  AMRAAM  program  office.  The 
Webster  Engineering  contract  contained  an  organizational  conflict 
of  interest  clause  that  should  have  prevented  Veda  from 
supporting  the  operational  tests  because  Veda  was  already 
supporting  the  AMRAAM  program  office.  This  condition  was 
previously  described  under  the  Navy  systems. 
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Using  the  Navy  Test  Center's  contract  with  McLaughlin,  the  Air 
Force  Operational  Test  Agency  tasked  the  contractor  to  provide 
support  for  the  Air  Force  initial  and  follow-on  operational  tests 
of  AMRAAM  test  firing  programs.  Tasks  included  providing 
technical  and  analytical  expertise  for  planning,  designing, 
establishing  support  requirements,  and  conducting/reporting  of 
operational  tests  involving  the  F-16/AMRAAM  and  the  F-15/AMRAAM 
weapon  systems.  The  McLaughlin  contract  did  not  have  an 
organizational  conflict  of  interest  clause  that  related  to  the 
contractor  not  supporting  operational  tests  and  development. 

MILSTAR  System.  The  MILSTAR  Joint  Program  Office, 
collocated  with  the  Air  Force  Systems  Command,  Space  Systems 
Division,  has  acquisition  and  program  management  responsibility 
for  the  satellite  and  mission  control  segments  of  MILSTAR.  The 
Air  Force  Systems  Command,  Electronic  Systems  Division,  has 
responsibility  for  the  Air  Force  terminal  segment.  The  Army  and 
Navy  will  procure  their  own  service-unique  terminals,  and  the 
terminals  interoperations  will  be  managed  by  the  Joint  Terminal 
Program  Office  under  the  direction  of  the  MILSTAR  Joint  Program 
Office.  The  Air  Force  Operational  Test  Agency  was  responsible 
for  operational  tests  of  the  Air  Force  terminal,  and  mission 
control  segments  for  the  MILSTAR  system. 


We  determined  that  three  contractors:  Booz-Allen  &  Hamilton, 
Incorporated  (Booz-Allen);  BDM;  and  Science  Applications 
International  Corporation  (SAIC)  participated  in  the  development 
and  the  operational  tests  of  the  MILSTAR  system  (Appendix  E). 
Details  of  the  tasks  the  contractors  performed  are  as  follows. 

Development.  The  Joint  Terminal  Program  Office  has  a 
contract  with  Booz-Allen  that  was  awarded  in  November  1986  to 
provide  system  engineering,  configuration  management,  integrated 
logistics,  test  and  evaluation,  development  and  analysis  for 
MILSTAR  segments,  and  MILSTAR  terminal  interoperability  test 
planning  and  support.  As  of  October  1990,  the  total  cost  of  the 
contract  was  about  $12  million. 

The  MILSTAR  Air  Force  terminal  production  contractor,  Rockwell 
International  Corporation,  subcontracted  with  BDM  in  October  1990 
to  perform  Radiation  Lot  Acceptance  Testing  on  specific  parts  for 
the  Air  Force  MILSTAR  terminal  program  at  a  cost  of  about 
$296,000. 

The  MILSTAR  satellite  and  mission  control  segment  production 
contractor,  Lockheed  Missiles  and  Space  Company,  subcontracted 
with  SAIC  in  January  1989  to  conduct  specified  Electromagnetic 
Pulse  Analyses  for  the  MILSTAR  mission  control  element  system. 
As  of  June  1990,  the  cost  was  about  $355,000. 
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Operational  tests.  The  Air  Force  Operational  Test 
Agency  had  two  service  contracts  with  Booz-Allen  that  were 
awarded  in  February  1986  and  May  1989.  The  agency  issued  a  total 
of  five  task  orders  under  these  contracts  to  support  the  MILSTAR 
system.  The  contractor  was  tasked  to  develop  an  Operational  Test 
Nuclear  Assessment  Plan  for  the  MILSTAR  Air  Force  terminal  and 
mission  control  element.  The  tasks  were  to  support  the  initial 
operational  tests  of  the  system  and  to  execute  the  initial  phase 
of  the  operational  nuclear  survivability  assessment  plan  for  the 
terminal  in  support  of  the  low-rate  initial  production  decision 
and  the  multiservice  initial  operational  tests.  In  addition,  the 
contractor  was  to  support  the  survivability  portions  of  the  Air 
Force's  early  operational  assessment  and  initial  operational 
tests  for  the  mission  control  and  terminal  segments.  The 
estimated  cost  of  the  MILSTAR  task  orders  on  these  contracts  was 
$1.4  million. 

The  Air  Force  Operational  Test  Agency  also  had  a  services 
contract  with  BDM  that  was  awarded  in  July  1985  to  provide 
technical  support  to  the  agency.  From  1986  to  1989  the  Test 
agency  issued  four  task  orders  to  BDM  in  support  of  the  MILSTAR 
system.  BDM  was  required  to  provide  technical  effort  to  support: 
the  development  of  the  MILSTAR  initial  operational  tests  and 
multiservice  test  plans;  the  development  of  the  Air  Force 
Terminal  low-rate  initial  production  operational  assessment  plan 
and  the  baseline  set  of  test  scenarios  for  the  terminal 
interoperability  initial  operational  tests;  and  the  development 
of  the  terminal  operational  test  event  descriptions,  flight 
profiles,  and  data  collection  methods  used  to  collect  appropriate 
operational  test  data.  The  estimated  cost  of  the  MILSTAR  tasks 
on  this  contract  was  about  $1.34  million. 

The  Air  Force  Operational  Test  Agency  also  had  a  services 
contract  with  SAIC  that  was  awarded  in  July  1989  to  support  the 
survivability  portion  of  the  early  operational  assessment  and  the 
initial  operational  tests  of  the  MILSTAR  Satellite.  Specific 
tasks  required  SAIC  to  support  MILSTAR  test  execution,  test 
reporting,  methodology  development,  and  special  investigations 
and  analyses.  The  estimated  cost  of  these  tasks  was  $500,000. 

The  organizational  conflict  of  interest  clauses  in  the 
three  contracts  should  have  precluded  these  contractors  from 
supporting  the  Air  Force  Operational  Test  Agency  in  these 
instances. 

Radar  System.  The  System  Program  Office  collocated  with  the 
Air  Force  Systems  Command,  Electronic  Systems  Division,  has 
acquisition  and  program  management  responsibility  for  the  Radar 
system.  The  Air  Force  Operational  Test  Agency  was  responsible 
for  planning,  conducting,  and  reporting  initial  operational  tests 
for  the  Radar  system. 
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One  contractor,  SAIC,  participated  in  the  development  and  the 
operational  tests  of  the  Radar  system  (Appendix  F).  Details  of 
the  tasks  SAIC  performed  are  as  follows. 

Development.  The  Radar  system's  prime  contractor. 
General  Electric  Company,  subcontracted  with  SAIC  in 
November  1986  to  perform  an  engineering  and  economic  study/ 
analysis  to  determine  the  most  cost-effective  method  of 
transferring  processed  radar  data  from  the  receive  site  to  the 
operations  center  for  the  West  Coast  Radar  system  at  a  total  cost 
of  about  $290,000. 

Operational  tests.  The  Air  Force  Operational  Test 
Agency  tasked  SAIC  in  December  1989  to  provide  analytical  support 
for  test  planning,  execution,  reporting,  software  development, 
and  simulation/modeling  activities  for  the  Radar  system  East 
Coast  Radar  Station  initial  operational  tests.  SAIC  was  also 
tasked  to  maintain  a  system  to  reduce  and  format  data  collected 
during  initial  operational  tests  for  analysis  by  agency.  The 
total  cost  of  the  SAIC  tasks  was  about  $1.6  million. 

Services  Contractors  that  Participated  in  Development 

Program  managers  were  not  required  to  maintain  a  list  of  services 
contractors  that  participated  in  the  development  of  systems. 
Consequently,  contracting  officers  did  not  know  which  services 
contractors  should  be  excluded  from  the  operational  tests,  and  it 
was  difficult  for  the  Director  and  the  Test  agencies  to  identify 
when  the  same  services  contractors  were  used  for  both  operational 
tests  and  development  of  the  systems. 

Waivers  For  Pse  of  Services  Contractors 


The  audit  was  based  on  a  strict  interpretation  of  10  U.S.C.  2399, 
which  prohibits  the  same  services  contractor  from  supporting  both 
operational  tests  and  development  of  the  same  system.  Some  of 
the  instances  identified  in  the  report,  where  the  same  services 
contractors  were  involved  in  both  areas,  related  to  only  minor 
involvement  by  the  contractor  in  either  the  operational  tests  or 
the  development  of  the  system.  These  violations  of  the  statute 
probably  had  no  impact  on  the  impartiality  or  independence  of  the 
operational  tests.  Even  though  the  Director  did  not  request  any 
waivers  during  FY  1991,  the  statute  does  provide  that  the 
Director  may  waive  the  limitation  of  the  use  of  services 
contractors  if  the  Director  determines  in  writing  that  sufficient 
steps  have  been  taken  to  ensure  the  impartiality  of  the 
contractor  in  providing  the  services.  The  Inspector  General, 
DoD,  is  required  to  review  the  Director's  waivers  and  make  an 
assessment  in  its  semiannual  report.  However,  there  are  no  such 
provisions  for  the  Test  agencies  to  obtain  waivers  and  use  the 
same  services  contractors  under  similar  conditions. 
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Organizational  Conflicts  of  Interest 

Federal  Acquisition  Regulation  (FAR),  subpart  9.5, 
"Organizational  Conflicts  of  Interest,"  prescribes 
responsibilities,  general  rules,  and  procedures  to  identify, 
evaluate,  and  resolve  organizational  conflicts  of  interest.  The 
FAR  also  provides  examples  to  assist  contracting  officers  in 
applying  the  rules  and  procedures  to  individual  contracting 
situations.  The  regulation  states  that  an  organizational 
conflict  of  interest  exists  when  the  contractor  obtains  an  unfair 
competitive  advantage  or  when  its  objectivity  would  be  impaired 
without  some  restriction  of  future  activities.  However,  the 
organizational  conflicts  of  interest  guidance  in  the  FAR  does  not 
address  the  issue  of  using  the  same  services  contractors  to 
support  both  the  development  and  the  operational  tests  of  a 
system.  Our  review  showed  that  the  Test  agency's  contracting 
officers  used  different  conflicts  of  interest  clauses. 
Furthermore,  because  there  was  not  a  standard  organizational 
conflict  of  interest  clause,  the  clauses  used  did  not  always 
preclude  services  contractors  from  supporting  both  the 
development  and  the  operational  tests  of  the  same  systems. 

RECOMMENDATIONS,  MANAGEMENT  COMMENTS,  AND  AODIT  RESPONSE 

1.  We  recommend  that  the  Director  of  Operational  Test  and 
Evaluation  in  conjunction  with  the  Deputy  Director,  Defense 
Research  and  Engineering  (Test  Evaluation): 

a.  Amend  DoD  Directive  5000.2,  "Defense  Acquisition 
Management  Policies  and  Procedures"  to  require  program  managers 
to  maintain  a  list  of  all  advisory  and  assistance  services 
contractors  and  subcontractors  that  participated  in  the 
development ,  production,  or  testing  for  major  Defense  acquisition 
systems. 

Director  of  Operational  Test  and  Evaluation  comments.  The 
Director  stated  that  DoD  instruction  5000.2  could  be  amended  to 
require  that  each  program  manager  maintain  a  list  of  services 
contractors  used  during  development,  production,  or  testing. 
Contracting  officers  for  the  Operational  Test  Agencies  would  then 
be  required  to  exclude  those  services  contractors  from  the 
operational  tests. 

Audit  response.  We  request  that  management  provide  a 
completion  date  for  this  action  when  responding  to  the  final 
report.  In  the  draft  audit  report,  we  recommended  that  the 
Test  and  Evaluation  Master  Plan  be  used  to  identify  the 
services  contractors  supporting  development  and  operational 
tests.  Based  on  managements  response,  we  deleted  that 
portion  of  the  recommendation  from  the  final  report. 
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b.  Propose  legislation  that  would  allow  the  Operational  Test 
Agencies  to  obtain  waivers  from  the  Director  of  Operational  Test 
and  Evaluation  to  use  the  same  advisory  and  assistance  services 
contractors  that  participated  in  the  development,  production,  or 
testing  to  also  support  the  operational  test  and  evaluation  under 
justifiable  conditions  when  sufficient  steps  have  been  taken  to 
ensure  the  impartiality  of  the  contractor  services. 

Director  of  Operational  Test  and  Evaluation  comments.  The 
Director  nonconcurred  with  the  recommendation  and  commented  that 
initiating  legislation  at  this  point  appeared  to  be  premature 
because  he  had  not  been  made  aware  of  any  difficulty  in  obtaining 
services  contractors  due  to  legislative  constraint. 

Army  comments.  OPTEC  concurred  with  the  recommendation  and 
commented  that  services  contractors,  by  virtue  of  their  earlier 
involvement  in  the  development  of  the  program  and  the  highly 
specialized  nature  of  the  services  provided,  often  had  the 
technical  knowledge  and  expertise  that  could  not  otherwise  be 
acquired  and  were  sometimes  imperative  to  the  operational  tests. 
OPTEC  thought  they  should  be  able  to  use  the  same  services 
contractors  in  selected  instances  when  the  contractors 
impartiality  could  be  demonstrated  and  that  approval  authority 
for  the  waivers  should  reside  within  the  Service  requiring  the 
support . 

Air  Force  comments.  AFOTEC  concurred  with  the 
recommendation,  supported  the  proposal  for  legislation,  and 
thought  that  legislation  to  clarify  the  existing  law  in  this  area 
would  be  extremely  beneficial.  AFOTEC  also  commented  that  the 
audit  was  based  on  an  overly  strict  interpretation  of  the  use  of 
contractors  in  operational  tests. 

Audit  response.  Based  on  input  from  the  Army  and  Air  Force, 
we  request  that  the  Director  of  Operational  Test  and 
Evaluation  reconsider  its  position  about  the  provisions  for 
waivers  when  responding  to  the  final  report.  The  audit  was 
based  on  a  strict  interpretation  of  10  U.S.C.  2399,  which 
prohibits  the  same  services  contractors  from  supporting  both 
operational  tests  and  development  of  the  same  systems.  Some 
of  the  instances  identified  in  the  report,  where  the  same 
services  contractors  were  involved  in  both  areas,  related  to 
only  minor  involvement  by  the  contractor  in  either  the 
operational  tests  or  the  development  of  the  system. 
Although  these  were  violations  of  the  statute,  they  probably 
had  no  impact  on  the  impartiality  or  independence  of  the 
operational  tests.  In  addition,  there  may  also  be  instances 
where  contractors  with  a  certain  technical  expertise  that 
participated  in  the  development  of  the  system  could  be 
beneficial  to  the  operational  tests. 
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c.  Develop  a  standard  organizational  conflict  of  interest 
clause  that  precludes  advisory  and  assistance  services 
contractors  and  subcontractors  from  participating  in  development f 
production,  or  testing  and  operational  test  and  evaluation  for 
the  same  systems  unless  a  waiver  is  obtained. 

Director  of  Operational  Test  and  Evaluation  comments.  The 
Director  partially  concurred  with  the  recommendation  and 
commented  that  he  would  work  to  develop  such  a  clause,  but 
without  the  provision  for  a  waiver. 

Audit  response.  We  request  that  management  provide  a 
completion  date  for  this  action  when  responding  to  the  final 
report . 

2.  We  recommend  that  the  Director  of  Defense  Procurement  direct 
the  Defense  Acquisition  Regulations  Council  to  evaluate  the 
conflict  of  interest  clause  developed  by  the  Director  of 
Operational  Test  and  Evaluation  and  take  appropriate  action  to 
include  the  clause  related  to  advisory  and  assistance  services 
support  for  operational  test  and  evaluation  to  the  Defense 
Federal  Acquisition  Regulation  Supplement. 

Director  of  Defense  Procurement  comments.  The  Director  of 
Defnese  Procurement  concurred  with  the  recommendation  as  stated. 

3.  We  recommend  that  the  Commanders  of  the  Military  Departments' 
Operational  Test  Agencies: 

a.  Insert  organizational  conflict  of  interest  clauses  in 
existing  and  future  contracted  advisory  and  assistance  services 
contracts  that  preclude  contractors  and  subcontractors  from 
participating  in  development,  production,  or  testing  and  in 
operational  test  and  evaluation  for  the  same  systems. 

b.  Direct  contracting  officers  to  formally  notify 
contractors  of  the  provisions  for  impartial  contracted  advisory 
and  assistance  services  in  10  O.S.C.  2399. 

c.  Direct  contracting  officers  to  enforce  the  provisions  in 
10  D.S.C.  2399  when  contractors  or  subcontractors  are 
participating  in  the  development ,  production,  or  testing  and  in 
the  operational  test  and  evaluation  of  the  same  systems . 

d.  Report  the  material  internal  control  weakness  of  using 
the  same  advisory  and  assistance  services  contractors  to  support 
operational  test  and  evaluation  that  participated  in  development , 
production,  or  testing,  and  track  the  status  of  corrective 
actions  taken  until  the  problems  noted  are  resolved. 

Army  comments.  OPTEC  concurred  with  Recommendations  A. 3. a., 
A.3.b.,  and  A.3.c.  and  commented  that  contracts  currently 
included  a  clause  as  indicated.  OPTEC  also  commented  that 
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contracts  transferred  to  its  control  as  a  result  of  the  Test  and 
Evaluation  reorganization  would  be  amended  to  include  the 
clause.  OPTEC  nonconcur red  with  Recommendation  A.3.d.  and 
commented  that  the  same  services  contractors  were  not  being  used 
to  support  two  different  activities  resulting  in  a  conflict  of 
interest . 

Audit  response.  We  are  providing  a  detailed  response  to 
OPTEC  comments  to  Recommendation  A.3.d.  as  Appendix  U.  Based 
on  that  response/  we  request  that  management  reconsider  its 
position  when  responding  to  the  final  report. 

Navy  comments.  OPTEVFOR  concurred  with  each  of  the 
recommendations  and  commented  it  presently  had  an  excellent 
organizational  conflict  of  interest  clause  that  was  included  in 
each  analytical  support  contract.  OPTEVFOR  stated  that  its 
contract  specialist  would  have  a  conference  to  formally  notify 
the  prospective  contractor  of  the  provisions  prior  to  award  of 
all  future  contracts.  In  addition,  OPTEVFOR  commented  that  they 
were  actively  taking  action  to  remove  the  existing  conflict  of 
interest  with  regards  to  the  Webster  Engineering,  subcontractor 
Veda,  Inc.,  and  that  a  new  contract  would  be  awarded  by 
August  1992. 

Air  Force  comments.  AFOTEC  concurred  with  Recommendations 
A. 3. a.,  A.3.b. ,  and  A.3.c.  and  commented  that  instructions  would 
be  provided  to  appropriate  contracting  activities  regarding  the 
specific  clause  to  be  used  in  each  contract.  AFOTEC  also  stated 
that  instructions  would  be  provided  to  appropriate  contracting 
activities  to  notify  contractors  of  the  provisions  of  10  O.S.C. 
2399  and  stated  that  it  is  presently  taking  action  to  enforce  the 
provisions  and  will  continue  to  do  so. 

AFOTEC  also  concurred  with  Recommendation  A.3.d.  and  commented 
that  they  would  report  material  internal  control  weaknesses  in 
those  cases  where  they  felt  adequate  safeguards  to  mitigate 
conflict  of  interest  could  not  be  ensured.  However,  AFOTEC  did 
not  agree  with  the  concept  of  reporting  all  circumstances  where 
the  same  contractors  supported  both  operational  tests  and 
development,  when  adequate  safeguards  were  in  place. 

Audit  response.  Those  instances  where  the  same  services 
contractors  supported  both  operational  tests  and  development 
should  be  reported  as  material  internal  control  deficiencies 
because  they  are  in  violation  of  the  law.  Unless  legislation 
is  passed  that  permits  the  Test  agencies  to  obtain  waivers  to 
use  the  same  services  contractors  that  supported  development 
under  certain  justifiable  conditions,  even  those  instances 
where  AFOTEC  believes  adequate  safeguards  were  in  place 
should  be  reported.  We  request  that  management  reconsider 
its  position  when  responding  to  the  final  report. 
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STATUS  OF  RECOMMENDATIONS 


Number  Addressee 


Concur/ 

Nonconcur 


Response  Should  Cover: 
Proposed  Completion 
Action  Date 


Related 

Issues* 


A.l.a.  DOT&E 


X  IC 


A.l.b.  DOT&E 


X 


X 


X 


A  •  1  •  C  • 

DOT&E 

A. 2. 

DDP 

No 

A. 3. a. 

OPTEC 

NO 

OPTEVFOR 

No 

AFOTEC 

No 

A. 3 .b. 

OPTEC 

No 

OPTEVFOR 

No 

AFOTEC 

No 

A. 3 . c. 

OPTEC 

No 

OPTEVFOR 

No 

AFOTEC 

No 

A.3.d. 

OPTEC 

X 

OPTEVFOR 

No 

AFOTEC 

X 

X  IC 

Further  Response  Required 

Further  Response  Required 
Further  Response  Required 
Further  Response  Required 

Further  Response  Required 
Further  Response  Required 
Further  Response  Required 

Further  Response  Required 
Further  Response  Required 
Further  Response  Required 

X  X  IC 

Further  Response  Required 

X  X  IC 


*  IC  =  material  internal  control  weakness 
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B.  COST-EFFECTIVENESS  OF  SERVICES  CONTRACTS 


The  Director  and  the  Test  agencies  used  repeated  and  extended 
services  contracts  that  were  not  as  cost-effective  as  using 
in-house  civilian  staffs  to  support  operational  tests  for  major 
Defense  acquisition  systems.  This  occurred  because  the  Director 
and  the  Test  agencies  lacked  sufficient  in-house  civilian  staff 
to  adequately  perform  their  mission.  As  a  result,  the  Director 
and  the  Test  agencies  depended  on  services  contractors  for 
program  continuity  and  the  "corporate  knowledge"  needed  to  plan, 
analyze,  and  report  operational  tests;  spent  over  $44  million 
annually  for  contractor  assistance  that  was  not  as  cost-effective 
as  developing  an  in-house  capability  to  perform  the  work;  and 
used  the  same  contractors  to  support  the  operational  tests  that 
participated  in  the  development  of  systems  (Finding  A).  We 
estimate  that  the  Director  and  the  Test  agencies  could  save  about 
$26.1  million  for  FY's  1992  through  1996  by  reducing  their 
services  contracts  60  percent  and  developing  an  in-house 
capability  to  support  operational  tests.  In  addition,  the  Test 
agencies  did  not  report  or  control  these  services  contracts  as 
contracted  advisory  and  assistance  services  (CAAS). 

DISCUSSION  OF  DETAILS 


Background 

DoD  Directive,  4205.2,  January  27,  1986,  "DoD  Contracted  Advisory 
and  Assistance  Services  (CAAS),"  establishes  policy,  assigns 
responsibilities,  and  prescribes  procedures  for  planning, 
managing,  evaluating,  and  reporting  services  contracts.  The 
Directive  defines  CAAS  as  those  services  acquired  directly  by  DoD 
from  nongovernmental  sources  to  support  or  improve  agency  policy 
development  or  decision  making,  or  to  Support  or  improve  the 
management  of  organizations  or  the  operation  of  weapon  systems, 
equipment,  and  components.  There  are  four  CAAS  categories: 
individual  experts  and  consultants;  studies,  analyses,  and 
evaluations;  management  support  services;  and  engineering  and 
technical  services. 

The  Directive  states  that  services  contracts  may  be  used  when 
suitable  in-house  capability  is  unavailable  and  cannot  be 
developed  in  time  to  meet  the  needs  of  the  DoD  Component 
concerned,  or  development  of  an  in-house  capability  would  not  be 
cost-effective  because  the  special  skills  or  expertise  are  not 
required  full  time.  The  directive  also  states  that  services 
shall  be  obtained  on  an  intermittent  or  temporary  basis,  as 
required,  and  that  repeated  or  extended  services  arrangements 
shall  not  be  entered  into  except  under  extraordinary 
circumstances.  The  directive  prohibits  the  use  of  services 
contracts  to  bypass  or  undermine  personnel  ceilings,  pay 
limitations,  or  competitive  employment  procedures. 


23 


Repeated  and  Extended  Services  Contracts 

The  Director  and  the  Test  agencies  used  repeated  and  extended 
services  contracts  to  support  operational  tests.  We  found  that 
the  same  contractors  or  contractor  personnel  were  consistently 
providing  support  to  the  Test  agencies  year  after  year.  For 
example,  Veda  had  been  supporting  operational  tests  of  Navy 
fighter  weapon  systems  since  1971.  Consequently,  the  contractors 
provided  the  program  continuity  and  gained  the  "corporate 
knowledge"  needed  to  plan,  analyze,  and  report  operational 
tests.  The  tasks  the  contractors  performed  are  described  in 
Finding  A.  The  Director  and  the  Test  agencies  spent  over 
$44  million  on  services  contracts  in  FY  1990.  A  breakout  of  the 
amounts  the  Director  and  the  Test  agencies  spent  on  these 
contracts  in  FY  1990  and  a  brief  summary  of  the  extent  of 
services  used  over  the  past  few  years  follow. 
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The  Director  spent  a  total  of  about  $31.1  million  for  contractor 
services  during  FY's  1987  through  1989.  About  $18.5  million  was 
contracted  with  private  companies,  and  $12.6  million  was 
contracted  with  the  Institute  for  Defense  Analyses,  a  DoD 
Federally  Funded  Research  and  Development  Center.  In  FY  1990, 
the  Director  spent  almost  $10  million  for  support  services,  with 
the  Institute  performing  about  90  percent  of  the  work. 

The  Army  Operational  Test  Agency  spent  a  total  of  about 
$28.4  million  for  contractor  services  during  FY's  1989  and  1990; 
however,  the  Army  Test  Agency  was  only  the  evaluator  for 
operational  tests.  The  Command  that  conducted  the  operational 
tests  also  spent  unknown  amounts  for  their  own  services 
contractors.  For  example,  the  Test  and  Experimentation  Command, 
Experimentation  Center,  was  the  initial  operational  test  and 
evaluation  tester  for  the  Pedestal  Mounted  Stinger  and  Line  of 
Sight-Forward-Heavy  systems.  The  Experimentation  Center  had  a 
services  contract  awarded  in  June  1985  with  a  base  year  and 
4  option  years  at  a  total  price  of  about  $71.7  million.  Under 
the  contract,  the  Experimentation  Center  tasked  its  services 
contractor  to  develop  portions  of  the  test  evaluation  plan, 
detailed  test  plan,  and  test  report  for  the  initial  operational 
tests  of  the  two  systems.  The  Test  and  Experimentation  Command  is 
now  part  of  the  new  Army  Operational  Test  and  Evaluation 
Command.  Consequently,  the  Army's  newly  established  Operational 
Test  Agency  will  spend  more  for  services  to  support  operational 
tests  in  FY  1991  and  future  years  than  the  amounts  identified  in 
this  report. 

The  Navy  Operational  Test  Agency  spent  a  total  of  about 
$15.3  million  for  contractor  services  during  FY's  1987  through 
1989  and  over  $6  million  during  FY  1990. 

The  Air  Force  Operational  Test  Agency  spent  a  total  of  about 
$24.8  million  for  contractor  services  during  FY's  1988  and  1989 
and  over  $16  million  during  FY  1990. 

Services  Contracts  Were  Not  Cost-effective 

We  determined  that  the  services  contracts  used  to  support 
operational  tests  were  not  as  cost-effective  as  developing  an 
in-house  capability  to  perform  the  work.  Although  it  may  not  be 
practical  to  bring  100  percent  of  these  services  in-house  because 
of  fluctuations  in  work  load,  it  would  be  cost-effective  to  bring 
a  significant  amount  of  the  effort  in-house.  We  determined  that 
the  Director  and  the  Test  agencies  were  spending  an  average  of 
between  21.3  percent  and  37.9  percent  more  for  contractor 
personnel  than  the  costs  of  comparable  civilian  Government 
employees  (Appendix  H) .  We  estimate  that  the  Director  and  the 
Test  agencies  could  save  about  $8  million  annually  by  reducing 
their  services  contracts  60  percent  and  developing  an  in-house 
capability  to  plan,  analyze,  and  report  operational  tests.  Our 
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total  estimated  savings  of  $26.1  million  for  FY's  1992  through 
1996  were  based  on  a  gradual  yearly  reduction  of  services 
contracts  by  10  percent,  25  percent,  40  percent,  60  percent,  and 
60  percent,  respectively,  and  replacing  those  contractor 
personnel  with  comparable  civilian  Government  employees. 

We  developed  FY  1990  estimated  hourly  costs  for  various  grade 
levels  of  civilian  Government  personnel  and  added  additional 
burdens  such  as  retirement,  medicare,  health  insurance,  and 
fringe  benefits  to  those  costs  (Appendix  I).  We  compared  those 
costs  to  the  FY  1990  contractor  and  subcontractor  fully  burdened 
hourly  costs  negotiated  on  the  services  contracts.  Examples  of 
our  analysis  of  contractor  costs  and  comparable  civilian 
Government  costs  for  the  Director  and  Test  agencies  are  provided 
as  Appendixes  J  through  M,  respectively.  Further  details  follow. 

o  The  Director  was  spending  about  31  percent  more  for 
contractor  services  than  the  cost  of  performing  the  work  in-house 
With  comparable  civilian  Government  employees.  We  estimate  that 
the  Director  can  save  a  total  of  about  $6.1  million  during 
FY's  1992  through  1996  by  reducing  its  services  contracts  about 
$19.4  million  and  spending  about  $13.3  million  for  comparable 
civilian  Government  employees  (Appendix  N). 

o  The  Army  Operational  Test  Agency  was  spending  about 
22  percent  more  for  contractor  services  than  the  cost  of 
performing  the  work  in-house  with  comparable  civilian  Government 
employees.  We  estimate  that  the  Army  can  save  a  total  of  about 
$5.1  million  during  FY's  1992  through  1996  by  reducing  its 
services  contracts  about  $23.4  million  and  spending  about 
$18.3  million  for  comparable  civilian  Government  employees 
(Appendix  0) . 

o  The  Navy  Operational  Test  Agency  was  spending  about 
21  percent  more  for  contractor  services  than  the  cost  of 
performing  the  work  in-house  with  comparable  civilian  Government 
employees.  We  estimate  that  the  Navy  can  save  a  total  of  about 
$2.4  million  during  FY's  1992  through  1996  by  reducing  its 
services  contracts  about  $11.8  million  and  spending  about 
$9.4  million  for  comparable  civilian  Government  employees 
(Appendix  P) . 

o  The  Air  Force  Operational  Test  Agency  was  spending  about 
38  percent  more  for  contractor  services  than  the  cost  of 
performing  the  work  in-house  with  comparable  civilian  Government 
employees.  We  estimate  that  the  Air  Force  can  save  a  total  of 
about  $12.5  million  during  FY's  1992  through  1996  by  reducing  its 
services  contracts  about  $31.8  million  and  spending  about 
$19.8  million  for  comparable  civilian  Government  employees 
(Appendix  Q) . 
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Eliminate  Organizational  Conflicts  of  Interest 

As  described  in  Finding  A,  the  same  services  contractors  that 
supported  operational  tests  also  participated  in  the  development 
of  the  systems.  These  services  contractors  had  supported  the 
systems  for  many  years  and  had  provided  the  program  continuity 
and  the  "corporate  knowledge"  needed  to  plan,  analyze,  and  report 
the  operational  tests.  However,  by  bringing  the  support  for 
operational  tests  in-house  at  the  Test  agencies,  DoD  would  gain 
valuable  knowledge  about  the  systems  being  tested  and  also 
eliminate  the  potential  for  any  organizational  conflict  of 
interest  in  this  area. 

Services  Contracts  Were  Not  Controlled  or  Reported  as  CAAS 

The  Test  agencies  did  not  control  or  report  their  services 
contracts  to  support  operational  tests  as  CAAS.  The  Test 
agencies  stated  that  these  contracts  were  exempt  from  CAAS 
controls  and  reporting  requirements  based  on  current  CAAS 
exemptions.  However,  we  reviewed  the  CAAS  exemptions  and  could 
not  find  any  that  in  our  opinion  would  exempt  these  services  from 
being  reported  and  controlled  as  CAAS.  Further,  the  Director 
reported  and  controlled  its  services  contracts  as  CAAS. 
Consequently,  there  were  only  limited  controls  over  the  services 
contracts  used  by  the  Test  agencies  to  support  operational 
tests.  Some  of  the  additional  CAAS  controls  include  an 
explanation  of  what  is  to  be  procured;  a  clear,  explicit 
justification  of  the  need  and  expected  benefit  of  the  services; 
and  review  and  approval  by  the  DoD  Component  Director  for  CAAS. 

Office  of  Management  and  Budget  Guidance 

On  March  25,  1991,  the  Office  of  Management  and  Budget  provided 
budget  guidance  to  DoD  and  stated  that  by  September  1991, 
necessary  steps  should  be  taken  to  ensure  that  adequate  staffing 
is  available  for  the  performance  of  inherently  governmental 
functions.  The  guidance  further  stated  that  DoD  should  review 
its  internal  instructions  to  ensure  that  managers  had  adequate 
guidance  on  which  to  base  decisions  to  contract  for  advisory  and 
assistance  services  and  recognize  those  functions  that  must  be 
performed  by  Federal  officials  and  employees.  The  Office  of 
Management  and  Budget  stated  that  it  would  assist  DoD  in  this 
effort,  where  appropriate. 

RECOMMENDATIONS,  MANAGEMENT  COMMENTS,  AND  AODIT  RESPONSE 

1.  We  recommend  that  the  Director  of  Operational  Test  and 
Evaluation  and  the  Commanders  of  the  Military  Departments' 
Operational  Test  Agencies  in  conjunction  with  the  Assistant 
Secretary  of  Defense  (Force  Management  and  Personnel); 
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a.  Determine  in-house  civilian  personnel  requirements  needed 
to  perform  their  mission. 

b.  Make  appropriate  funding  adjustments  in  budget  request  to 
gradually  hire  the  additional  civilian  personnel. 

c.  Establish  a  goal  to  reduce  the  use  of  advisory  and 
assistance  services  contractors  by  60  percent  over  the  next 
5  years. 

Director  of  Operational  Test  and  Evaluation  and  the  Deputy 
Assistant  Secretary  of  Defense  (Requirements  &  Resources) 
commentsT  The  Director  and  the  Deputy  Assistant  Secretary 
concurred  with  Recommendation  B.l.a.  and  commented  that  the 
Office  of  the  Assistant  Secretary  of  Defense  (Force  Management 
and  Personnel)  issues  annual  guidance  to  the  DoD  Components  to 
review  all  of  their  manpower  requirements.  The  guidance 
addresses  military  and  civilian  manpower  and  states  that  the 
Department's  manpower  should  represent  "the  most  cost  efficient 
and  flexible  mix  of  manpower  necessary  to  support  mission 
accomplishment."  This  manpower  requirements  review  is 
accomplished  as  part  of  the  Department's  annual  budget  review. 

The  Director  and  the  Deputy  Assistant  Secretary  nonconcurred  with 
Recommendations  B.l.b.  and  B.l.c.  and  commented  that  if  the 
annual  review  of  manpower  requirements  determined  that  there  was 
a  continuing,  long-term  requirement  for  additional  in-house 
civilian  manpower  and  that  there  should  be  a  shift  from 
contractor  personnel  to  in-house  Government  employees,  then  the 
organization's  total  obligational  authority  should  have 
sufficient  funding  to  finance  these  additional  in-house 
personnel.  (If  funding  for  contracts  goes  down,  then  there  will 
be  additional  funding  available  to  pay  in-house  salaries.)  The 
Department  uses  the  in-house  and  contractor  civilian  work  forces 
as  fungible  resources.  The  Director  also  commented  that  the 
proposed  percentage  reduction  in  contractor  services  was 
"arbitrary  and  capricious,"  with  no  basis  in  fact  and  no 
consideration  of  organizational  and  operational  requirements. 

Audit  response.  The  Director  and  Deputy  Assistant  Secretary 
concurred  with  the  Recommendation  A.l.a.,  but  they  did  not 
describe  a  plan  or  date  when  reductions  will  be  initiated. 
Although  we  did  not  review  those  contractor  services  procured 
by  the  Director  prior  to  FY  1987,  the  Director  has  spent 
about  $10  million  per  year  for  contractor  services  in 
FY's  1988  through  1991.  Unless  the  Director  has  determined 
that  his  mission  requirements  will  be  greatly  reduced  in 
future  years,  there  appears  to  be  a  continuing  long-term 
requirement  for  contractor  support.  We  request  that 
management  provide  additional  information  when  responding  to 
the  final  report  and  determine  the  in-house  civilian 
personnel  requirements  needed  to  perform  its  mission. 
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Based  on  comments  from  the  Director  and  the  Deputy  Assistant 
Secretary,  we  revised  recommendations  B.l.b.  and  B.l.c.  We 
believe  the  proposed  reduction  in  contractor  services  was  a 
reasonable  estimate.  The  Director's  budget  for  FY's  1987 
through  1989  was  $34.7  million  and  over  88  percent  of  it,  or 
$31  million,  was  spent  for  contractor  assistance.  In 
FY  1990,  the  Director  spent  almost  $10  million  for  contractor 
assistance.  We  request  that  management  reconsider  its 
position  when  responding  to  Recommendations  B.l.b.  and  B.l.c. 
in  the  final  report. 

Army  comments.  OPTEC  concurred  with  Recommendations  B.l.a., 
B.l.b.,  and  B.l.c.  and  commented  that  they  were  anxious  to  bring 
more  work  in-house,  and  concurrently  become  less  reliant  upon 
contractor  support.  However,  OPTEC  civilian  end  strength  was 
reduced  by  278  spaces  in  response  to  DMRD  936,  and  further 
reductions  are  planned  for  the  next  fiscal  year.  Concurrently, 
the  level  of  test  and  evaluation  activity  has  remained  constant. 
OPTEC  stated  that  it  has  been  the  Command's  objective  for 
sometime  to  reduce  the  level  of  contractor  support  and  commented 
that  a  reduction  of  60  percent  in  contractual  support  over  the 
next  5  years  was  a  reasonable  goal.  However,  OPTEC  thought  it 
was  impossible  to  speculate  whether  this  reduction  would  generate 
the  estimated  savings. 

Audit  response.  Congress  enacted  Public  Law  98-473  in 
FY  1985  to  remove  civilian  employment  end-strength  ceilings, 
and  in  FY  1986  DoD  adopted  a  ceiling  free  management  policy. 
Annual  DoD  manpower  guidance  to  the  DoD  Components  has  stated 
that  civilian  workforce  levels  should  be  matched  to  funded 
work  loads  and  mission  requirements.  Further,  the  Navy 
demonstrated  that  it  is  possible  to  replace  contractor 
support  with  in-house  civilian  personnel.  Based  on  a  1988 
Navy  Inspector  General  report,  the  Navy  initiated  a  6-year 
effort  to  recruit  3,178  additional  full-time  personnel  to 
provide  in-house  engineering  and  management  support.  The 
estimated  savings  should  be  substantiated  once  the  contractor 
effort  is  brought  in-house. 

Navy  comments.  OPTEVFOR  nonconcurred  with  Recommendations 
B.l.a.,  B.l.b.,  and  B.l.c.  and  commented  that  its  programs  do  not 
require  continual  analytical  support.  OPTEVFOR  stated  that  there 
are  times  when  extensive  analytical  support  is  required  and  other 
times  when  no  analytical  involvement  is  required. 

Audit  response.  We  agree  that  there  are  fluctuations  in  the 
amount  of  analytical  support  required  and  that  any  given 
program  may  not  require  continual  support;  however,  there 
appears  to  be  a  continual  need  for  a  baselevel  amount  of 
analytical  support.  This  baselevel  support  is  needed  to 
provide  the  program  continuity  and  "corporate  knowledge" 
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needed  to  plan,  analyze,  and  report  operational  tests.  For 
example,  OPTEVFOR's  detachment  at  the  Pacific  Missile  Test 
Center  must  rely  on  contractor  support  to  help  plan,  analyze, 
and  report  operational  tests  because  it  has  no  civilian  staff 
to  perform  this  function.  These  contractor  employees  work  on 
multiple  programs  and  are  providing  what  amounts  to  continual 
support.  We  recommend  that  management  reconsider  its 
position  when  responding  to  the  final  report. 

Air  Force  comments.  AFOTEC  concurred  with  Recommendation 
B.l.a.  and  recommended  an  in-depth  analysis  of  the  costs  and 
benefits  associated  with  reduced  levels  of  contractor  dependence. 
AFOTEC  further  commented  that  its  extended  services  contracts 
were  technical  and  scientific-type  contracts  that  provided  its 
functional  elements  specialized  expertise. 

AFOTEC  nonconcurred  with  Recommendations  B.l.b.  and  B.l.c. 
pending  further  analysis  of  the  costs  and  benefits  associated 
with  reduced  levels  of  contractor  dependence. 

Audit  response.  We  recognize  that  some  of  AFOTEC' s 
specialized  technical  and  scientific-type  work  may  be  more 
effectively  performed  by  specialized  services  contractors. 
However,  in-house  civilian  employees  can  also  effectively 
perform  tasks  in  these  areas  and  in  general  work  areas  such 
as  concept  development,  test  planning,  test  execution,  data 
management,  modeling/simulation,  and  survivability  analysis. 
AFOTEC  did  not  state  when  it  would  initiate  the  analyses  of 
the  cost  and  benefits  associated  with  contractor  support.  We 
request  that  AFOTEC  provide  this  information  in  response  to 
the  final  report. 

2.  We  recommend  that  the  DoD  Director  of  Contracted  Advisory  and 
Assistance  Services,  in  conjunction  with  the  Comptroller  of  the 
Department  of  Defense: 

a.  Include  the  use  of  services  contracts  to  support 
operational  test  and  evaluation  in  the  ongoing  revision  to  the 
definition  for  contracted  advisory  and  assistance  services. 

b.  Provide  guidance  that  requires  the  Military  Departments' 
Operational  Test  Agencies  to  report  and  control  their  advisory 
and  assistance  services  contracts  to  support  operational  test  and 
evaluation  as  contracted  advisory  and  assistance  services. 

Director  of  DoD  Contracted  Advisory  and  Assistance  Services 
comments .  The  Director  of  DoD  CAAS  nonconcurred  with  the 
recommendation  and  commented  that  he  could  not  require  that  Test 
agencies  report  those  advisory  and  assistance  services  efforts 
that  were  currently  exempted/excluded  from  the  definition  of 
CAAS.  The  current  definition  provides  specific  exclusions  that 
can  be  reasonably  applied  to  contractor  support  used  by  the  Test 
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agencies.  However,  there  are  ongoing  initiatives  to  strengthen 
the  management,  identification,  and  reporting  of  CAAS.  A  major 
task  was  to  develop  an  "easier  to  apply  CAAS  definition"  and 
ensure  that  it  was  consistently  applied  throughout  DoD.  The 
improved  definition  of  CAAS  will  be  included  in  the  revised  DoD 
Directive  4205.2,  by  October  1,  1991. 

Audit  response.  Based  on  comments  from  the  Director  of  DoD 
CAAS,  we  revised  Recommendation  B.2.  For  many  years  Congress 
has  been  interested  in  the  DoD  Components'  use  of  CAAS,  an 
area  often  perceived  as  vulnerable  to  abuses  such  as  conflict 
of  interest,  favoritism,  and  unreasonable  costs.  Further, 
the  issue  of  an  organizational  conflicts  of  interest 
involving  the  operational  tests  and  development  of  a  major 
Defense  acquisition  system  is  critical.  The  audit  concluded 
that  there  were  organizational  conflicts  of  interests  and 
unreasonable  costs  associated  with  the  services  contractors 
used  by  the  Test  agencies.  Consequently,  we  believe  the 
improve  definition  of  CAAS  must  not  exempt/exclude  these 
advisory  and  assistance  services  used  by  the  Test  agencies. 
Therefore,  based  on  the  improved  definition  of  CAAS,  we 
request  that  management  reconsider  its  position  when 
responding  to  the  final  report. 

STATUS  OF  RECOMMENDATIONS 


_  Response  Should  Cover: _ 

Concur/  Proposed  Completion  Related 

Number  Addressee  Nonconcur  Action  Date  Issues* 


B. 1 .a. 

DOT&E 

X 

X 

X 

M 

ASD( FM&P) 

X 

X 

X 

M 

OPTEC 

No 

Further 

Response 

Required 

OPTEVFOR 

X 

X 

X 

M 

AFOTEC 

X 

M 

B.l.b. 

DOT&E 

X 

M 

ASD( FM&P ) 

X 

X 

X 

M 

OPTEC 

No 

Further 

Response 

Required 

OPTEVFOR 

X 

X 

X 

M 

AFOTEC 

X 

X 

X 

M 

B .  1 .  c . 

DOT&E 

X 

X 

X 

M 

ASD( FM&P ) 

X 

X 

X 

M 

OPTEC 

NO 

Further 

Response 

Required 

OPTEVFOR 

X 

X 

X 

M 

AFOTEC 

X 

X 

X 

M 

B. 2 .a. 

DCAAS 

X 

X 

X 

IC 

B.2.b. 

DCAAS 

X 

X 

X 

IC 

*  m  =  monetary  benefits;  IC  =  material  internal  control  weakness 
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-  Summary  of  Cost  Difference  in  using  Services 

Contracts  Versus  In-House  Civilian  Resources 
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-  Comparison  of  Contractor  Costs  Versus  In-House  Costs 

for  Webster  Engineering  Contract  N00123-89-D-0039 

-  Comparison  of  Contractor  Costs  Versus  In-House  Costs 

for  SAIC  Contract  F29601-89-C-0070 

-  Director  of  Operational  Test  and  Evaluation, 

Projected  Savings  From  Performing  Work  In-House 
For  FY's  1992-1996 

-  Army  Operational  Test  and  Evaluation  Agency, 

Projected  Savings  From  Performing  Work  In-House 
For  FY's  1992-1996 

-  Navy  Operational  Test  and  Evaluation  Force, 

Projected  Savings  From  Performing  Work  In-House 
For  FY's  1992-1996 

-  Air  Force  Operational  Test  and  Evaluation  Center, 

Projected  Savings  From  Performing  Work  In-House 
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APPENDIX  G:  LISTING  OF  ACRONYMS  AND  ABBREVIATIONS 


ADABD . Army  Air  Defense  Artillery  Board 

ADASCH . Army  Air  Defense  Artillery  School 

AFOTEC..... . Air  Force  Operational  Test  and  Evaluation  Center 

AFSC . Air  Force  Systems  Command 

AMC. !!!!!!.!! . Army  Materiel  Command 

ARMTE* ! ! ! ! . Army  Materiel  Test  and  Evaluation  Directorate 

. . ASI  Systems  International 

ATI  1 . . . . Advanced  Technology  Incorporated 

BOOZ-ALLEN . . . . . Booz,  Allen  &  Hamilton  Incorporated 

. . The  BDM  Corporation 

CAs! . . . . . .  .CAS  Incorporated 

COLSa!  !!!!..!..!! . . . . COLSA  Incorporated 

DOT&E . Director  of  Operational  Test  and  Evaluation 

DSC . . ...Dynaspan  Services  Company 

oynetics  . . . Dynetics  Incorporated 

ENTER. , . ! . . . , . Entek  Corporation 

Essex!*.... . Essex  Corporation 

ESD. . . . . . Electronic  Systems  Division 

FDT&E . . . Force  Development  Test  and  Evaluation 

FOT&e! . Follow-on  Operational  Test  and  Evaluation 

. . General  Electric  Company 

Ida!!!! . institute  for  Defense  Analysis 

. . Initial  Operational  Test  and  Evaluation 

. . Joint  System  Program  Office 

JTPO . . . .....Joint  Terminal  Program  Office 

LMSC . Lockheed  Missiles  &  Space  Company 

MACA . Management  Assistance  Corporation  of  America 

MICOM . . . . . . . . .  .Army  Missile  Command 

MRC . . . McLaughlin  Research  Corporation 

. . Munitions  Systems  Division 

NADC . * . Naval  Air  Development  Center 

NAS . Native  American  Services  Associates 

NAVAIR . .....Naval  Air  Systems  Command 

OPTEVFOR . . . Navy  Operational  Test  and  Evaluation  Force 

OT&E . Operational  Test  and  Evaluation 

OTEA . Army  Operational  Test  and  Evaluation  Agency 

PMTC . Pacific  Missile  Test  Center 

PRC....... . . . Planning  Research  Corporation 

. . ..Physical  Science  Laboratory 

raven . . . . Raven  Incorporated 

ROCKWELL. . Rockwell  International  Corporation 

SAIC . Science  Applications  International  Corporation 

SIMSUM . . . . . SIMSUM  Incorporated 

. . ......System  Program  Office 

SSD . . . Space  Systems  Division 

SVERDRUP. . Sverdrup  Technology  Incorporated 


47 


APPENDIX  G:  LISTING  OF  ACRONYMS  AND  ABBREVIATIONS  (Cont'd) 

Tac . . Tactical  Air  Command 

tec!  !!!!!!!!... . TEXCOM  Experimentation  Center 

TECOM . . . Army  Test  and  Evaluation  Command 

TEXCOM . Army  Test  and  Experimentation  Command 

TRADOC . . . Army  Training  and  Doctrine  Command 

UIE . United  International  Engineering 

YEDA . . . . . .Veda  Incorporated 

VERAC/BALL . . . Ball  Systems  Engineering 

VX-4 . Air  Test  and  Evaluation  Squadron  Four 

WEBSTER . . Webster  Engineering 
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APPENDIX  K:  COMPARISON  OF  CONTRACTOR  COSTS  VERSUS  IN-HOUSE  COSTS  FOR  BOH  CONTRACT  HDA9Q3-B8-0-0018 
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APPENDIX  R;  SUMMARY  OF  POTENTIAL  MONETARY  AND  OTHER  BENEFITS 
~  RESULTING  FROM  THE  AUDIT  ~~~ 


Recommendation 
Reference _ 


Description  of  Benefit 


A.l.a.  Internal  Control. 

Requires  Program  Management 
Offices  and  Test  Agencies 
to  maintain  a  list  of 
services  contractors 
for  each  system. 

A.l.b.  Program  Results. 

DOT&E  propose  Legislation  to 
allow  Test  Agencies  to  obtain 
waivers  to  use  the  same 
contractors  that  supported 
Development  to  Support 
Operational  Tests 

A.l.c.  Internal  Control. 

DOT&E  develop  a  standard 
organizational  conflicts  of 
interest  clause  for  advisory 
and  assistance  services 
contracts  for  Operational  tests. 


A. 2.  Compliance  with  Regulations 

and  Laws. 

Revises  the  DFARS  to  require 
the  addition  of  the  standard 
organizational  conflicts  of 
interest  clause  in  all  contracts 
for  Operational  tests. 


A. 3. a.  Internal  Control. 

Requires  the  Test  Agencies  to 
include  a  conflicts  of 
interest  clause  in  each 
advisory  and  assistance 
services  contract  for 
Operational  tests 

A.3.b.  Internal  Control. 

Requires  Test  Agencies  to 
modify  existing  contracts  and 
notify  contractors  of 
existing  clauses. 


Amount  and/or 
Type  of  Benefit 

Nonmonetary. 


Nonmonetary. 


Nonmonetary . 


Nonmonetary. 


Nonmonetary. 


Nonmonetary. 
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RESULTING  FROM  THE  ADDIT  (Cont'd) 


Recommendation 

Reference _ Description  of  Benefit 


A. 3 . c . 


A.3.d. 


B.l.a. 


Internal  Control. 

Requires  Test  Agencies  to 
enforce  existing  conflicts  of 
interests  clauses  when 
contractors  do  not  adhere  to 
the  contract  conditions. 

Internal  Control. 

Requires  the  Test  Agencies 
to  report  the  material  internal 
control  weakness  of  using  the 
same  contractors  to  support 
Operational  tests  that 
supported  development. 

Program  Results. 

DOT&E  and  the  Test  Agencies 
determine  civilian  personnel 
requirements  needed  to  perform 
their  mission. 


B.l.b. 


B.l.c. 


B.2. 


Program  Results. 

DOT&E  and  the  Test  Agencies 
request  funding  to  hire  the 
additional  personnel. 

Program  Results. 

DOT&E  and  Test  Agencies 
reduce  the  use  of  contractor 
services  to  support  Operational 
tests  60  percent  over  the  next 
5  years. 

Internal  Control. 

Requires  the  Test  Agencies 
to  control  and  report 
contractor  services  to 
support  Operational  tests 
as  CAAS. 


Amount  and/or 
Type  of  Benefit 

Nonmonetary. 


Nonmonetary. 


Funds  put  to  better 
use  of  $26.1  million 
(DOT&E  $6.1  million. 
Army  $5.1  million. 
Navy  $2.4  million, 
and  Air  Force 
$12.5  million)  for 
FY's  1992  through 
1996. 

Benefits  included 
in  Recommendation 
B.l.a. 


Benefits  included 
in  Recommendation 
B.l.a. 


Nonmonetary. 
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Office  of  the  Secretary  of  Defense 

Director,  Operational  Test  and  Evaluation,  Washington,  DC 
Department  of  the  Army 

U.S.  Army  Operational  Test  and  Evaluation  Command,  Alexandria,  VA 
U.S.  Army  Missile  Command,  Redstone  Arsenal,  AL 
U.S.  Army  Communications  and  Electronics  Command, 

Fort  Monmouth,  NJ 

U.S.  Army  Ammunitions  and  Munitions  Command, 

Picatinny  Arsenal,  NJ 

U.S.  Army  Test  and  Evaluation  Command,  Aberdeen 
Proving  Grounds,  MD 

U.S.  Army  Training  and  Doctrine  Command,  Fort  Monroe,  VA 
U.S.  Army  White  Sands  Missile  Range,  White  Sands 
Missile  Range,  NM 

U.S.  Army  Air  Defense  Artilery  School,  Fort  Bliss,  TX 

U.S.  Army  Field  Artilery  School,  Fort  Sill,  OK 

U.S.  Army  Air  Defense  Artilery  Board,  Fort  Bliss,  TX 

U.S.  Army  Field  Artilery  Board,  Fort  Sill,  Ok 

U.S.  Army  Test  and  Experimentation  Center,  Fort  Ord,  CA 

Department  of  the  Navy 

Navy  Operational  Test  and  Evaluation  Force,  Norfolk,  VA 

Chief,  Naval  Operations  Washington,  DC 

Naval  Air  Systems  Command,  Washington,  DC 

Naval  Space  and  Warfare  Command,  Washington,  DC 

Naval  Regional  Contracting  Center  Detachment,  Long  Beach,  CA 

Naval  Air  Development  Center,  Warminister,  PA 

Naval  Weapons  Center,  China  Lake,  CA 

Pacific  Missile  Test  Center,  Point  Mugu,  CA 

Air  Test  and  Evaluation  Site  1,  Patuxent  River,  MD 

Air  Test  and  Evaluation  Site  4,  Point  Mugu,  CA 

Air  Test  and  Evaluation  Site  5,  China  Lake,  CA 

Department  of  the  Air  Force 

Air  Force  Operational  Test  and  Evaluation  Center, 

Kirtland  AFB,  NM 

Air  Force  Systems  Command,  Andrews  Air  Force  Base,  MD 
Tactical  Air  Command,  Langley  Air  Force  Base,  VA 
Aeronautical  Systems  Division,  Wright-Patterson 
Air  Force  Base,  OH 

Electronic  Systems  Division,  Hanscom  Air  Force  Base,  MA 
Munitions  Systems  Division,  Eglin  Air  Force  Base,  FL 
Space  Systems  Division,  Los  Angeles  Air  Force  Base,  CA 
USAF  Tactical  Air  Warfare  Center,  Eglin  Air  Force  Base,  FL 
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Department  of  the  Air  Force  (Cont'd) 

USAF  Tactical  Fighter  Weapons  Center,  Nellis  Air  Force  Base,  NV 
Air  Force  Flight  Test  Center,  Edwards  Air  Force  Base,  CA 
57TH  Fighter  Weapons  Wing,  Nellis  Air  Force  Base,  NV 
554th  Operations  Support  Wing,  Nellis  Air  Force  Base,  NV 
AFOTEC  Detachment  2,  Eglin  Air  Force  Base,  FL 
AFOTEC  Detachment  3,  Nellis  Air  Force  Base,  NV 
AFOTEC  Detachment  5,  Edwards  Air  Force  Base,  CA 
Special  Management  Office  (LANTIRN),  Langley  AFB,  VA 
Joint  Terminal  Program  Office  (MILSTAR),  Arlington,  VA 

OTHER 

Institute  for  Defense  Analysis,  Alexandria,  VA 
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Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  For  Acquisition 
Comptroller,  Department  of  Defense 

Assistant  Secretary  of  Defense  (Force  Management  and  Personnel) 

Director  of  Defense  Procurement 

Director  of  Operational  Test  and  Evaluation 

Director  of  Contract  Advisory  and  Assistance  Services 

Director,  Defense  Acquisition  Regulations  System 

Department  of  the  Army 

Secretary  of  the  Army 

Assistant  Secretary  of  the  Army  (Financial 
Management ) 

Commander,  Army  Operational  Test  and  Evaluation  Command 
Department  of  the  Navy 
Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial 
Management) 

Commander,  Navy  Operational  Test  and  Evaluation  Force 
Department  of  the  Air  Force 
Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial 
Management  and  Comptroller) 

Commander,  Air  Force  Operational  Test  and  Evaluation  Center 
Non-DoD  Activities 


Office  of  Management  and  Budget 

General  Accounting  Office,  NS I AD,  Technical  Information  Center 
Congressional  Committees: 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Senate  Ranking  Minority  Member,  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
House  Ranking  Minority  Member,  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 
House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 
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Other 


The  Honorable  Harry  Reid,  United  States  Senate 

The  Honorable  Barbara  Boxer,  House  of  Representatives 

The  Honorable  Robert  J.  Lagomarsino,  House  of  Representatives 

The  Honorable  Elton  Gallegly,  House  of  Representatives 
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Army  comment.  OPTEC  commented  that  the  involvement  of  CAS  in 
contractual  support  to  the  program  office  as  well  as  to  the 
independent  operational  tester  for  the  Line  of  Sight-Forward- 
Heavy  system  appears  to  offer,  at  least  the  perception  of  an 
organizational  conflict  of  interest.  OPTEC  stated  the 
observation  was  based  entirely  on  information  provided  in  the 
audit  report,  which  they  were  not  able  to  independently  confirm 
or  disprove.  Further,  three  of  the  four  OTEA  task  orders  for 
support  were  awarded  before  the  effective  date  of  the 
legislation. 

Audit  response.  As  stated  in  the  audit  report,  CAS  provided 
technical  support  to  the  program  manager  for  the  Line  of 
Sight-Forward-Heavy  system.  The  technical  support  had  been 
ongoing  since  at  least  February  1988,  in  the  areas  of  system 
integration,  logistics,  production  engineering,  and  program 
management  in  support  of  the  continuing  system  acquisition 
process. 

Although  3  of  the  4  OTEA  task  orders  were  awarded  before  the 
effective  date  of  the  legislation,  OTEA  delivery  order  28  was 
issued  on  February  8,  1990,  to  obtain  technical  services  in 
support  of  OTEA's  participation  in  the  initial  operational 
test  and  evaluation  of  the  Line  of  Sight-Forward-Heavy 
system.  The  contractor  was  required  to  provide  technical 
support  to  the  independent  operational  evaluator  in  several 
areas,  review  test  documentation,  manage  and  analyze  data  in 
support  of  the  operational  assessment  and  independent 
evaluation  report,  and  provide  technical  assistance  in 
preparation  of  evaluation  briefings  and  reports.  The  total 
cost  of  the  delivery  order  issued  to  BDM  was  about 
$1.6  million;  however,  CAS,  as  a  subcontractor  to  BDM, 
provided  about  $500,000  of  the  effort.  Using  CAS  to  support 
the  program  manager  during  development  of  the  system  and  to 
support  the  operational  tests  for  the  same  system,  represents 
an  organizational  conflict  of  interest  and  a  violation  of 
10  U.S.C.  2399. 

Army  comment.  OPTEC  nonconcur red  that  any  organizational 
conflict  of  interest  occurred  when  COLSA  provided  contractual 
support  to  the  program  manager  and  the  TRADOC  ADASCH  in  the 
conduct  of  FDT&E  activities  for  the  Pedestal  Mounted  Stinger 
system.  OPTEC  contends  that  FDT&E  often  involves  (a  surrogate) 
system  rather  than  the  system  to  be  fielded  and  is  relied  upon  to 
verify  logistics  supportability ,  doctrine,  concepts  and 
organization.  This  testing  is  distinctly  different  from 
operational  testing  and  evaluation  as  defined  in  10  U.S.C.  2399. 
OPTEC  further  comments  that  the  law  was  intended  to  prevent 
contractor  involvement  in  the  Initial  Operational  Test  and 
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Evaluation  of  a  system  since  it  normally  supports  a  production  or 
low-rate  production  of  a  system  (See  Appendix  G  for  acronyms) 

Audit  response.  The  TRADOC  ADASCH  prepared  the  independent 
evaluation  plan  and  the  independent  evaluation  reports  for 
FDT&E  phases  I  and  II  of  the  Pedestal  Mounted  Stinger 
system.  After  thoroughly  reviewing  these  reports,  we 
concluded  that  FDT&E  was  basically  early  operational  tests. 
Further,  the  Test  and  Evaluation  Master  Plan  for  both  the 
Pedestal  Mounted  Stinger  system  and  the  Line  of  Sight- 
Forward-Heavy  system  identified  FDT&E  as  an  operational 
test.  Consequently,  using  the  same  services  contractor  that 
supported  the  program  manager  to  support  FDT&E  does  appear  to 
be  an  organizational  conflict  of  interest. 

As  described  in  the  independent  evaluation  plan,  the  purpose 
and  scope  of  FDT&E  phase  II  was  to  develop,  refine,  and 
validate  selected  operational  concepts,  critical  to  the 
deployment  and  use  of  the  Pedestal  Mounted  Stinger  system. 
The  operational  concepts  must  be  finalized  and  certified  as 
ready  for  test  prior  to  advancing  to  the  Initial  Operational 
Test  and  Evaluation.  The  initial  operational  test 
effectiveness  and  suitability  measures  of  performance  were 
assessed  to  provide  objective  data  to  support  the  evaluation 
of  the  FDT&E  phase  II  issues  and  criteria. 

The  Critical  Operational  Issue  tested  during  FDT&E  phase  II 
was  whether  the  Pedestal  Mounted  Stinger  system  provide  low- 
altitude  air  defense  for  mobile  and  stationary  critical 
assets  while  operating  outside  direct  fire  and  observed 
indirect  fire  ranges.  Other  operational  evaluation  or 
suitability  issues  included  whether  representative  soldiers 
perform  their  operator,  maintainer,  and  tactical  tasks; 
whether  reliability,  availability,  and  maintainability  of  the 
system  support  the  line-of-sight  rear  component  operational 
mission  profile;  and  whether  the  planned  logistics  support 
concept  for  the  system  was  adequate  to  support  operational 
requirements.  FDT&E  phase  II  also  utilized  production  fire 
units. 

We  compared  the  operational  evaluation  and  suitability  issues 
tested  during  FDT&E  phase  II,  to  the  issues  tested  during  the 
initial  operational  test  of  the  Pedestal  Mounted  Stinger  and 
found  that  they  were  almost  identical. 

Army  comments .  OPTEC  commented  that  BDM  was  awarded  OTEA 
delivery  order  12  on  January  6,  1989,  and  that  the  work  was 
completed  on  November  26,  1989.  BDM  was  also  awarded  a  contract 
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to  develop  the  Performance  Analysis  Data  Base  by  the  developer  on 
December  27,  1989,  and  the  work  was  completed  on  November  26, 
1990.  OPTEC  stated  that  this  work  did  not  conflict  with  the  work 
performed  on  OTEA  delivery  order  12. 

BDM  was  also  awarded  OTEA  delivery  order  40  on  September  21, 
1990,  to  support  the  Stinger  system.  The  intended  subcontractor 
was  Coleman  Research  Corporation  (Coleman  Research)  who  reported 
a  possible  conflict  of  interest  in  accordance  with  the  terms  of 
the  contract.  BDM  in  turn  reported  this  to  the  contracting 
officer  servicing  OPTEC.  The  contracting  officer  formally 
notified  BDM  by  correspondence  on  November  28,  1990,  that  a 
possible  conflict  did  exist  with  its  subcontractor  and  disallowed 
Coleman  Research  involvement.  BDM  formally  notified  the 
contracting  officer  that  BDM  could  not  perform  the  work  on 
January  6,  1991. 

Audit  response.  BDM  was  issued  OTEA  delivery  order  12  to 
provide  technical  support  to  the  independent  operational 
evaluator  for  the  Pedestal  Mounted  Stinger  system.  The 
technical  support  included  the  review  of  test  documentation, 
data  analysis  in  support  of  the  operational  assessment  and 
the  independent  evaluation  report,  and  technical  assistance 
in  the  preparation  of  evaluation  briefings  and  reports.  The 
total  cost  of  the  delivery  order  was  over  $1  million.  Even 
though  BDM  used  a  subcontractor  to  perform  the  majority  of 
the  work,  BDM  had  overall  responsibility  for  the  work 
performed.  This  delivery  order  was  completed  before 
10  U.S.C.  2399  was  enacted;  however,  there  was  still  the 
potential  for  an  organizational  conflict  of  interest  if  BDM 
was  also  supporting  the  developer  of  the  Pedestal  Mounted 
Stinger  system. 

The  Army  Ballistic  Missile  Defense  Organization  (developer) 
issued  a  task  order  to  BDM  on  November  27,  1989,  to  provide 
system  analysis  and  development  effort,  which  included  test 
planning  and  test  support  activities;  data  analysis  and 
technical  advisory  responsibilities;  and  systems  integration/ 
performance  analysis  activities.  The  systems  integration/ 
performance  analysis  activities  included  further  analysis  of 
subsystem  preplanned  product  improvements  for  the  Pedestal 
Mounted  Stinger.  The  task  stated  that  Stinger  missile  and 
other  components  of  the  Pedestal  Mounted  Stinger  system  had 
been  examined  for  system  improvements  and  new  subsystem 
integration  under  a  previous  task.  The  new  task  sought  to 
expand  on  what  had  been  accomplished  and  further  analyze 
threat  targets  and  their  impact  on  system  development 
improvements.  The  previous  task  was  awarded  to  BDM  on 
June  22,  1988,  for  Pedestal  Mounted  Stinger  noncooperative 
target  recognition  analysis  and  integration  support.  The 
total  cost  of  this  task  was  about  $1.4  million. 
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Consequently,  BDM  was  working  on  this  development  task  prior 
to,  and  during  its  support  of  the  operational  tests  for  the 
Pedestal  Mounted  Stinger  system.  This  support  created  an 
organizational  conflict  of  interest. 

On  November  1,  1990,  Coleman  Research  advised  BDM  that  it  had 
received  a  task  to  support  the  Stinger  Program  Office,  which 
created  an  organizational  conflict  of  interest  relating  to 
the  work  required  for  OPTEC  on  delivery  order  40. 

On  November  16,  1990,  the  contracting  officer  determined  that 
Coleman  Research  had  a  potential  conflict  of  interest  and 
could  not  work  on  delivery  order  40.  However,  the 
organizational  conflict  of  interest  relating  to  the  support 
BDM  had  provided  to  the  developer  was  never  discussed. 

On  January  8,  1991,  BDM  also  notified  the  contracting  officer 
that  it  had  a  potential  conflict  of  interest  relating  to  the 
work  required  on  delivery  order  40.  BDM  stated  its 
Huntsville  office  had  supported  the  Stinger  Project  Office  in 
two  general  categories:  the  Stinger  Performance  Analysis 
Database  and  the  concept  definition/formulation  to  support 
out  year  planning  for  Pedestal  Mounted  Stinger  product 
improvement  proposals  and  preplanned  product  improvement. 
BDM  also  stated  the  concept  definition  work  had  been  going  on 
for  sometime,  and  was  specifically  related  to  the  Pedestal 
Mounted  Stinger.  Consequently,  both  BDM  and  Coleman  Research 
had  an  organizational  conflict  of  interest.  Therefore,  BDM 
should  never  have  accepted  delivery  order  40  or  billed  the 
Government  for  about  $8,000  of  work  before  the  remaining 
funds  were  deobligated. 

Army  comment.  OPTEC  commented  that  the  report  concluded  that 
MACA  had  an  organizational  conflict  of  interest  since  it 
supported  FDT&E  and  IOT&E  of  the  Pedestal  Mounted  Stinger  system. 

Audit  response.  The  audit  concluded  that  FDT&E  and  IOT&E 
were  both  operational  tests,  therefore,  MACA  could  support 
both  tests  without  any  conflict  of  interest. 

Army  comment.  OPTEC  commented  that  the  report  concluded  that 
services  contractors  had  an  organizational  conflict  of  interest 
since  they  supported  FDT&E  and  IOT&E  for  the  Line  of  Sight- 
Forward-Heavy  system. 

Audit  response.  Again,  the  audit  concluded  that  FDT&E  and 
IOT&E  were  Both  operational  tests,  therefore,  the  same 
contractors  could  support  both  of  these  tests  without  any 
conflict  of  interest. 
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Army  comment .  OPTEC  commented  that  contrary  to  what  the 
report  concluded,  with  regard  to  organizational  conflicts  of 
interest,  independent  testers  and  evaluators  have  made  major 
strides  in  learning  to  work  in  compliance  as  well  as  enforce  the 
new  legislation.  Contractors  are  carefully  screened  to  preclude 
any  perception  of  conflict.  In  the  case  of  the  Line  of  Sight- 
Forward-Heavy  system,  both  Coleman  Research  and  Martin  Marietta 
were  excluded  from  competition  for  this  reason. 

Audit  response.  As  previously  described,  the  contractors 
could  not  have  been  carefully  screened  if  CAS,  COLSA,  and  BDM 
supported  both  development  and  the  operational  tests  for  the 
Line  of  Sight-Forward-Heavy  system  or  the  Pedestal  Mounted 
Stinger  system. 
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PART  IV  -  MANAGEMENT  COMMENTS 


Director  of  Operational  Test  and  Evaluation 

Office  of  the  Assistant  Secretary  of  Defense  (Force  Management 
and  Personnel) 

Director  of  Defense  Procurement 
Department  of  the  Army 
Department  of  the  Navy 
Department  of  the  Air  Force 

Director  of  DoD  Contracted  Advisory  and  Assistance  Services 
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OPERATIONAL,  TEST 
AND  EVALUATION 


OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

WASHINGTON,  DC  20301-1700 


St  JUN891 


MEMORANDUM  FOR  INSPECTOR  GENERAL  ( CONTRACT  MANAGEMENT 
DIRECTORATE) 

SUBJECT:  Draft  Audit  Report  on  Consulting  Services  Contracts 
for  Operational  Test  and  Evaluation  (Project  No. 
OCH-5009 ) 


Pursuant  to  your  request,  subject  report  has  been  reviewed 
by  this  office.  Specific  comments  on  the  recommendations 
addressed  to  DOT&E  are  attached. 


Robert  C.  Duncan 
-Director 


Attachments : 
As  stated 
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1.  We  recommend  that  the  DOT&E,  in  conjunction  with  the  DDDR&E 

a.  Require  Program  Mananement  offices  and  OTA's  to  identify  all 
advisory  and  assistance  services  contractors  and  subcontractors  that 
participate  in  the  development,  production,  or  testing  and  in  the 
operational  test  and  evaluation  for  major  Defense  acquisition  systems 
in  the  system’s  TEMP. 

DOT&E  RESPONSE:  Partially  Concur.  DOT&E  recognizes  the  need  to 
preclude  potential  conflicts  of  interest.  However,  inasmuch  as  the 
Test  and  Evaluation  Master  Plan  is  only  updated  at  milestones,  we 
feel  that  the  requirement  could  be  more  appropriately  satisfied  by 
amending  DODD  5000,2.  It  should  require  that  each  program  manager 
maintain  a  list  of  advisory  assistance  service  contractors  used 
during  development,  production,  or  testing.  Operating  test  agency 
Contracting  Officers  would  then  be  required  to  ensure  that  those 
advisory  and  assistance  services  contractors  are  excluded  from 
operational  testing,  to  preclude  any  conflict  of  interest,  whether 
real  or  apparent. 

b.  Propose  legislation  that  would  allow  OTA's  to  obtain  waivers 
from  the  DOT&E  to  use  the  same  advisory  and  assistance  services 
contractors  that  participated  in  the  development,  production  or 
testing  to  also  support  the  operational  T&E  under  justifiable 
conditions  when  sufficient  steps  have  been  taken  to  ensure  the 
impartiality  of  the  contractor  services. 

pOT&E  RESPONSE:  Nonconcur.  Initiating  legislation  at  this  point 
appears  to  be  premature.  There  is  insufficient  data  to  support 
changing  the  present  law,  insofar  as  DOT&E  has  not  been  made  aware  of 
any  difficulty  in  obtaining  CAAS  due  to  this  legislative  constraint. 

c.  Develop  a  standard  organizational  conflict  of  interest  clause 
that  precludes  advisory  and  assistance  contractors  and  subcontractors 
from  participating  in  development,  production  or  testing,  and  OT&E 
for  the  same  systems  unless  a  waiver  is  obtained. 

DOT&E  RESPONSE:  Partially  Concur.  DOT&E  will  work  to  develop  such  a 
clause,  but  without  a  provision  for  waivers. 

2.  We  recommend  that  the  DOT&E  and  the  Commanders  of  the  Military 
Departments  OTA’s  in  conjunction  with  the  ASD  (Force  Management  and 
Personnel): 

a.  Determine  in-house  civilian  personnel  requirements  needed  to 
perform  their  mission. 

dot&e  RESPONSE:  Concur.  The  Office  of  the  Assistant  Secretary  of 
Defense  (Force  Management  &  Personnel)  issues  annual  guidance  to  the 
DoD  components  to  review  all  of  their  manpower  requirements.  This 
guidance  addresses  military  and  civilian  manpower  and  states  that  the 
Department’s  manpower  should  represent  "the  most  cost  efficient  and 


MANAGEMENT  COMMENTS:  DIRECTOR  OF  OPERATIONAL  TEST  AND  EVALUATION  (Cont’d) 


flexible  mix  of  manpower  necessary  to  support  mission 
accomplishmnet . *  This  manpower  requirements  review  is  accomplished 
as  part  of  the  Department’s  annual  budget  review. 

b.  Request  funding  to  gradually  hire  the  additional  civilian 
personnel : 

dot&e  response :  Nonconcur.  If  the  annual  review  of  manpower 
requirements  determines  that  there  is  a  continuing,  long  term 
requirement  for  additional  in-house  civilian  manpower  and  that  there 
should  be  a  shift  from  contractor  personnel  to  in-house  government 
employees,  then  the  organization’s  total  obligational  authority  (TOA) 
should  have  sufficient  funding  to  finance  these  additional  in-house 
personnel.  (If  funding  for  contracts  goes  down,  then  there  will  be 
additional  funding  available  to  pay  in-house  salaries.)  The 
Department  uses  the  in-house  and  contractor  civilian  work  forces  as 
fungible  resources. 

c.  Reduce  the  use  of  Advisory  and  Assistance  Service  Contractors 
by  60  percent  over  the  next  five  years. 

dot&e  response:  Nonconcur.  This  proposed  percentage  reduction  is 
arbitrary  and  capricious,  with  no  basis  in  fact  and  no  consideration 
of  organizational  and  operational  requirements.  The  Director 
Operational  Test  and  Evaluation  can  best  determine  if  a  civilian 
manpower  space  should  be  filled  by  an  in-house  employee  or  by  a 
contractor,  using  long  standing  DoD  policy  ‘that  the  work  force  should 
reflect  the  most  cost  efficient  mix  of  manpower  necessary  to  support 
mission  accomplishment.  The  Department  does  not  want  to  incur  the 
long  term  financial  obligations  associated  with  additional  in-house 
personnel  unless  there  is  a  long  term  need  for  these  people.  For 
short  term  requirements,  contractor  support  may  be  a  better  bargain. 
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FORCE  MANAGEMENT  AND  PERSONNEL) 


THE  OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 


VASHINCTON  D  C  20301-1000 


0 1  J'JL 


force  management 

AND  PERSONNEL 


MEMORANDUM  FOR  DIRECTOR,  CONTRACT  MANAGEMENT  DIRECTORATE, 
DEPARTMENT  OF  DEFENSE  INSPECTOR  GENERAL 

SUBJECT:  Draft  Audit  Report  on  Co"su^in3  |erY^®sM^on^*Cco0|?r 

Operational  Test  and  Evaluation  (Project  No.  OCH-5009) 

The  above  referenced  draft  audit  report  addressed  three  DoDIG 
recommendations  for  corrective  actions  that  the  Director  for 
Operational  Test  and  Evaluation  and  the  respective  : Military. 
Service  commanders  should  take  in  coordination  with  the  Office  of 
the  Assistant  Secretary  of  Defense  (Force  Management  &  Person¬ 
nel)  .  I  offer  the  following  comments  on  those  recommendations: 

DoDIG  Recommendation: 

a.  Determine  in-house  civilian  personnel  requirements  needed 
to  perform  their  mission. 

FM&P  Comment : 

Concur.  The  Office  of  the  Assistant  Secretary  of  Defense 
(Force  Management  &  Personnel)  issues  annual  guidance  to  the  DoD 
components  to  review  all  of  their  manpower  requirements.  This 
guidance  addresses  military  and  civilian  manpower  and  states  that 
the  Department's  manpower  should  represent  "the  most  cost  effi- 
cent  and  flexible  mix  of  manpower  necessary  to  support  mission 
accomplishment."  This  manpower  requirements  review  is  accom¬ 
plished  as  part  of  the  Department's  annual  budget  review. 

DoDIG  Recommendation: 

b.  Request  funding  to  gradually  hire  the  additional  civilian 
personnel . 

FMtP  Comment: 

Nonconcur.  If  the  annual  review  of  manpower  requirements 
determines  that  there  is  a  continuing,  long-term  requirement  for 
additional  in-house  civilian  manpower  and  that  there  should  be  a 
shift  from  contractor  personnel  to  in-house  government  employees, 
then  the  organization's  total  obligational  authority  (TOA)  should 
have  sufficient  funding  to  finance  these  additional  in-house 
personnel.  (If  funding  for  contracts  goes  down,  then  there  will 
be  additional  funding  available  to  pay  in-house  salaries.)  The 
Department  uses  the  in-house  and  contractor  civilian  work  forces 
as  fungible  resources. 
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DoDIG  Recommendation: 

c.  Reduce  the  use  of  advisory  and  assistance  service  con¬ 
tractors  by  60  percent  over  the  next  5  years. 

FM&P  Comment: 

Nonconcur.  This  proposed  percentage  reduction  is  arbitrary 
and  capricious,  with  no  basis  in  fact  and  no  consideration  of 
organizational  and  operational  requirements.  The  Director  of 
Operational  Test  and  Evaluation  can  best  determine  if  a  civilian 
manpower  space  should  be  filled  by  an  in-house  employee  or  by  a 
contractor,  using  long-standing  DoD  policy  that  the  work  force 
should  reflect  the  most  cost  efficient  mix  of  manpower  necessary 
to  support  mission  acomplishment .  The  Department  does  not  want 
to  incur  the  long-term  financial  obligations  associated  with 
additional  in-house  personnel  unless  there  is  a  long-term  need 
for  these  people.  For  short-term  requirements,  contractor  sup¬ 
port  may  be  a  better  bargain^ 

/  // 

Carl  J.  Dahlman 
Deputy  Assistant  Secretary 
(Requirements  &  Resources) 
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acquisition 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON,  DC  20301 


'JUL 


1991 


DP  /CPA 

MEMORANDUM  FOR  THE  OFFICE  OF  THE  INSPECTOR  GENERAL,  DEPARTMENT  OF 
DEFENSE 

SUBJECT:  Draft  Audit  Report  on  Consulting  Services  Contracts  for 

Operational  Test  and  Evaluation  (Project  No.  OCH-5009) 


My  staff  has  reviewed  your  draft  audit  report  on  consulting 
services  contracts  for  operational  test  and  evaluation.  As  a  result 
of  that  review,  the  following  comment  is  offered. 

Racomendation  2:  We  recommend  that  the  Director  of  Defense 
Procurement  direct  the  Defense  Acquisition  Regulatory  Council  to 
evaluate  the  conflict  of  interest  clause  developed  by  the  Director,  of 
Operational  Test  and  Evaluation  and  take  appropriate  action  to 
include  the  clause  related  to  advisory  and  assistance  services 
support  for  operational  test  and  evaluation  to  [sic]  the  Defense 
Federal  Acquisition  Regulation  Supplement. 

ddp  Position:  Concur.  When  the  Director  of  Operational  Test 
and  Evaluation  has  submitted  a  draft  of  a  new  Organizational  Conflict 
of  Interest  clause  to  my  office,  I  will  forward  it  to  the  Defense 
Acquisition  Regulation  Staff  for  review.  I  cannot  establish  a  time 
for  the  completion  of  that  review  until  the  draft  of  the  new  clause 
is  received. 

I  appreciate  the  opportunity  to  comment  on  this  draft  report. 

Eleanor  R.  Spector 
Director,  Defense  Procurement 
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DEPARTMENT  OF  THE  ARMY 
OfFICC  OF  THE  UNDER  •ECRETAMV 
WASHINGTON  DC  *0*10  0102 

2  7  JUN  *»i 


SAUS-OR 


memorandum  for  department  of  defense  inspector  general 

SUBJECT:  Draft  Audit  Report  on  Consulting  Services  Contracts 

for  Operational  Test  and  Evaluation  (Project  No.  OCH- 
5009) 


The  subject  draft  audit  report  has  been  reviewed  within  the 
Army.  We  do  not  concur  with  a  number  of  the  conclusions  and 
recommendations  resulting  from  the  audit. 

We  do  not  concur  that  the  Army  Operational  Test  and 
Evaluation  Command  compromised  desired  impartiality  and 
independence  by  using  the  same  service  contractor  to  support 
operational  tests  who  participated  in  the  development  of  the 
system.  There  is  no  evidence  to  suggest  that  test  assessments 
were  biased  or  that  systems  were  produced  and  deployed  with 
unknown  performance  limitations.  Enclosures  1  and  2  detail  the 
specific  basis  for  our  nonconcurrence. 

The  Test  and  Evaluation  Master  Plan  is  not  a  proper  vehicle 
for  establishing  an  internal  control  mechanism  for  managing 
contractor  support.  Enclosure  2  provides  the  rationale  for  this 
position . 

Although  we  agree  with  the  recommendation  to  reduce 
contractor  support  and  pursue  the  development  of  in-house 
capability,  we  see  this  to  be  in  opposition  to  the  ongoing 
efforts  to  reduce  personnel  strength  in  the  Department  of  the 
Army  and  Defense  as  a  whole.  However,  we  intend  to  continue 
efforts  to  strike  a  proper  balance.  This  area  is  further 
discussed  in  enclosure  3. 

We  nonconcur  in  the  recommendation  to  establish  operational 
test  support  services  contracts  under  the  auspices  and  reporting 
requirements  of  the  Contract  Advisory  and  Support  Services 
(CAAS)  guidelines.  It  is  our  opinion  that  this  would  be 
inconsistent  with  the  FAR.  The  basis  for  this  position  can  be 
found  in  enclosure  4. 
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-2- 

SAUS-OR 

SUBJECT:  Draft  Audit  Report  on  Consulting  Services  Contracts 
for  Operational  Test  and  Evaluation  (Project  No.  OCH- 
5009) 


We  appreciate  the  opportunity  to  review  the  draft  report 
and  trust  that  our  concerns  will  receive  due  consideration. 
Questions  can  be  directed  to  my  action  officer  for  this  reply, 
LTC  Walter  Kaminski,  USATEMA,  ATTN:  DASC-TE,  Rm  3C571  Pentagon, 
(703)  695-8995. 


Enel  Walter  W.  Hollis 

Deputy  Under  Secretary  of  the  Army 
'  (Operations  Research) 


CF: 

DA  OIG  ATTN:  SAIG-PA 

DOTfiE 

DDDRE(TE) 

ASA(RDA)  ATTN:  SFRD-KP 
CDR  OPTEC  ATTN:  CSTE-OP 
CDR  AMC  ATTN:  AMCIR-A 
DIR  MISMA  ATTN:  SFUS-MIS 
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Service  Contractors  Supporting  Operational  Tests  and 
Development 

Comments  on  Organizational  conflict  of  Interest 


1.  The  audit  addressed  four  major  Army  programs.  The  amount  of 
contract  activity  was  considerable,  and  the  audit  was  apparently 
in  some  detail.  However,  the  situation  described  in  2.c.  below 
appears  the  only  possibility  where  even  the  perception  an  OCI 
might  have  occurred.  The  facts  do  not  support  the  auditors  overall 
conclusion  that  test  "did  not  attain  the  desired  impartiality  and 
independence,  test  assessments  may  be  biased,  and  systems  may  be 
produced  and  deployed  with  unknown  performance  limitations. 11 

2.  Part  II  of  the  report  discusses  test  agencies  using  the  same 
service  contractors  to  support  operational  test  and  evaluation  for 
major  defense  acquisitions  that  participated  in  systems  development. 
Appendixes  A  and  B  of  the  report  illustrate  instances  of  alleged 
organizational  conflict  of  interest.  We  find,  however,  that: 

a.  Six  of  the  nine  indicated  contractors  were  involved  in  Force 
Development  Test  and  Evaluation  (FDTE)  and  either  systems 
development  or  OT&E  activities.  However,  neither  combination  of 
activities  constitutes  an  organizational  conflict  of  interest  (OCI) 
as  defined  in  10  U.S.C.  Section  2399  (See  para  la,  attachment  1). 

b.  One  OCI  was  attributed  to  the  fact  that  BDM  supported  the 
program  office  for  the  Stinger  Missile  System  as  well  as  the 
operational  test  and  evaluation  agency  for  the  same  weapon  system. 
However,  after  reviewing  circumstances,  it  seems  apparent  that  an 
OCI  did  not  occur.  (See  para  l.b. ,  attachment  1). 

c.  One  additional  OCI  situation  was  associated  with  the  Line  of 
sight  Forward  Heavy  Air  Defense  System.  CAS  Corporation  provided 
contractual  services  to  the  program  office  as  well  as  to  the 
operational  tester.  It  appears  an  OCI  may  have  occurred.  However, 
the  auditors  should  have  taken  into  consideration  that  all  the  work, 
with  the  exclusion  of  a  single  task  order  was  on  contract  prior  to 
the  effective  date  of  the  legislation  (see  para  l.e.,  attachment  1) . 

3.  Testers  and  evaluators  have  made  major  strides  in  complying 
with,  as  well  as  enforcing  the  new  legislation.  The  contracting 
officer  is  also  enforcing  it,  and  has  included  OCI  clauses  in  all 
OPTEC  contracts  as  recommended  by  the  auditors  in  subject  report. 

The  contractors  give  every  indication  they  are  monitoring  their 
activities  as  well  as  the  activities  of  their  subcontractors. 


Enclosure  1 


management  comments 


DEPARTMENT  OF  THE  ARMY  (Cont'd) 


1.  ORGANIZATIONAL  CONFLICT  OF  INTEREST 

«.  The  DODIG  report  contends  that  COLSA  Corporation  had  an 
organizational  conflict  (OCI)  because  COLSA  provided  contractual 
support  to  the  PM  and  subsequently  supported  the  TRADOC/TEXCOM/ 
ADABD  and  the  TRADOC  ADASCH  in  the  conduct  of  FDTfcE  activities. 

(1)  This  command  non-concurs  with  any  conclusion  or 
suggestion  that  the  situation  described  above  involves  OCI.  FDTE 
is  distinctly  different  from  operational  testing  and  evaluation 
as  defined  in  Title  10,  United  States  Code,  Section  2399.  FDTE 
is  relied  upon  to  verify  logistics  supportability,  doctrine, 
concepts  and  organization.  FDTE  provides  information  critical  to 
the  materiel  fielding  process,  and  the  sustainability  of  the 
system  after  fielding?  it  does  not  support  a  materiel  fielding 
decision.  Rather  it  can  occur  at  anytime  in  the  materiel 
development  process,  and  often  involves  (a  surrogate)  system 
rather  than  the  system  to  be  fielded. 

(2)  The  law  is  intended  to  prevent  contractor 
involvement  in  the  Initial  Operational  Test  and  Evaluation  (IOTE) 
of  a  System  since  the  IOTE  normally  supports  a  production  or  low 
rate  production  of  a  system.  The  contractor  clearly  was  not 
involved  in  the  IOTE.  The  actions  of  both  the  contractor  and  the 
Army  were  well  within  the  spirit  and  intent  of  pertinent  public 
lavs. 


b.  The  DODIG  report  contends  BDM  had  a  conflict  of  interest 
because  they  supported  the  PM  in  the  development  of  a  Performance 
Analysis  Data  Base,  and  also  supported  OPTEC  during  the 
operational  evaluation  of  Pedestal  Mounted  Stinger. 

(1)  Following  is  a  clearer  description  of  the 
contractors  involvement  with  the  government. 

a.  BDM  was  awarded  OTEA  Delivery  Order  12  to 
support  OPTEC  on  6  January  1989.  The  work  was  completed 
26  November  1989. 

fc.  BDM  was  awarded  the  contract  to  develop  the 
Performance  Analysis  Data  Base  on  27  December  1989  by  the 
developer.  The  work  was  completed  on  November  26,  1990.  The 
work  was  titled:  "Sensor  Suite  Analysis  for  Integration  and  test 
support  for  Theatre  Missile  Defense  Operations".  The  work  was  in 
support  of  Army  Space  Defense  Command.  BDM  is  a  CITA  contractor 
for  Army  Space  Defense  Command.  This  work  did  not  conflict  with 
the  work  performed  on  delivery  order  12  described  above. 

£.  BDM  was  awarded  OTEA  Delivery  Order  40  on 
21  September  1990.  They  were  to  provide  contractual  support  to 
OPTEC  on  Stinger  BMP.  The  intended  subcontractor  was  Coleman 
Research.  Coleman  reported  a  possible  conflict  of  interest  in 
accordance  with  the  terms  of  their  contract  with  BDM.  BDM  in 
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turn  reported  accordingly  to  the  contracting  officer  servicing 
OPTEC.  The  contracting  officer  formally  notified  BDM  by 
correspondence  on  28  November  1990  that  a  possible  conflict  did 
exist  and  disallowed  Coleman  involvement.  BDM  formally  notified 
the  contracting  officer  that  BDM  could  not  perform  the  work  on 
6  January  1990. 

(2)  The  circumstances  outlined  in  2.b.  above  serve  to 
illustrate  two  very  important  points: 

4.  A  conflict  of  interest  never  occurred  as 
indicated  in  the  DODIG  report. 

fe.  A  conflict  of  interest  clause  is  contained  in 
the  contract  as  the  DODIG  Acknowledged.  The  contract  contains  a 
second  clause  that  requires  the  OCI  to  be  flowed  down  to  subs 
intact.  Neither  the  prime  or  his  sub  ignored  the  OCI  provisions. 
On  the  contrary,  both  reported  the  possibility  of  a  perceived  OCI 
thus  allowing  the  contracting  officer  to  take  appropriate  action. 

c.  The  DODIG  report  concludes  that  MACA  had  an  OCI  since 
MACA  supported  the  TRADOC/TEXCOM/ADABD  during  FDT&E,  and  OPTEC 
during  IOT&E  of  PMS.  This  is  not  a  conflict  of  interest 
situation  for  the  same  reasons  discussed  in  paragraph  l.a. (1)  and 
(2)  above.  FDT&E  is  not  the  test  of  a  system,  it  does  not  support 
a  production  decision,  and  MACA  was  not  employed  by  the 
contractor  responsible  for  developing  PMS.  Similarly,  the  DODIG 
report  concludes  COLSA  Corporation  had  a  conflict  of  interest 
situation  because  COLSA  provided  contractual  support  to  the 
TRADOC/ADASCH  during  both  FDT&E  1  and  FDT&E  11.  A  conflict  of 
interest  situation  does  not  exist.  Both  T&E  programs  supported  a 
single  Army  activity  involved  in  the  materiel  fielding  process. 
The  legislation  does  not,  and  should  not  inhibit  the  contractors 
involvement  in  this  situation.  Neither  an  OCI  nor  the  appearance 
of  an  OCI  exists. 

d.  The  DODIG  report  draws  similar  conclusions  with  regard  to 
service  contracts  supporting  the  LINE  OF  SIGHT  FORWARD  HEAVY 
SYSTEM.  COLSA  Corporation  supported  FDT&E  1  and  11,  MACA 
Corporation  supported  FDT&E  1  and  IOT&E,  and  PRC  Corporation 
supported  FDT&E  11  and  IOT&E.  It  is  the  contention  of  this 
organization  that  an  OCI  condition  did  not  occur  for  reasons 
already  discussed.  It  appears  the  DODIG  has  improperly 
interpreted  the  legislation  or  does  not  fully  understand  the  test 
and  evaluation  process* 

e.  The  involvement  of  CAS  Corporation  in  contractual  support 
to  the  program  office  as  well  as  to  the  independent  operational 
tester  appears  to  offer,  at  least  the  perception  of  an  OCI.  This 
observation  is  based  entirely  on  information  provided  in  the 
DODIG  report  which  we  were  neither  able  to  independently  confirm 
or  disprove.  BDM  was  the  Prime  Contractor  in  this  instance. 

Their  contract  with  the  government  contained  a  conflict  of 
interest  clause.  The  contract  contains  a  second  clause  requiring 
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the  OCI  clause  to  be  flowed  down  in-tact  to  all  subs.  The  clause 
was  flowed  down  to  CAS ,  who  was  the  sub  in  this  instance,  however 
the  potential  for  a  conflict  was  never  reported.  The  DODIG 
correctly  points  out  that  the  Sub  Contractor  did  not  adhere  to 
the  clause.  However,  the  DODIG  should  take  into  consideration 
that  the  PM  Office  award  to  CAS  occurred  in  February  1988,  and 
December  1989.  The  award  to  the  test  board  was  in  March  1987. 
Legislation  prohibiting  contractor  involvement  in  system 
development,  and  operational  test  and  evaluation  was  added  to  the 
FV  90  Defense  Authorization  Act  with  an  effective  date  of  29 
November  1989.  Three  of  the  four  OTEA  task  orders  were  also 
awarded  before  the  effective  date  of  the  legislation. 

f.  Contrary  to  what  is  concluded  in  the  DODIG  Report,  with 
regard  to  OCI  and  the  circumstance  described  in  l.e.  above, 
independent  testers  and  evaluators  have  made  major  strides  in 
learning  to  work  in  compliance  as  well  as  enforce  the  new 
legislation.  Contractors  are  carefully  screened  to  preclude  any 
perception  of  OCI.  In  the  case  of  Line  of  Sight  Forward  Heavy 
both  Coleman  Research  and  Martin  Marietta  were  excluded  from 
competition  for  this  reason.  The  OCI  clauses  contained  in  all 
OPTEC  contracts  is  working  effectively,  and  the  contractors  are 
policing  their  own  activities  as  well  as  those  of  their  subs. 

The  Prime  in  this  instance  has  reported  three  potential  OCI  in 
the  past  year;  these  included  KE  ASAT,  CCTT  and  FAADS.  The 
contracting  officer  acted  immediately  to  prohibit  their 
involvement  in  each  instance. 

2.  Nonconcur  with  DODIG  statement  contained  in  the  report  that: 

a.  "Military  Departments*  Operational  Test  Agencies  used  the 
same  service  contractors  to  support  operational  test  for  major 
defense  acquisition  system  that  participated  in  the  development 
of  the  systems". 

b.  "Operational  test  did  not  attain  the  desired  impartiality 
and  independence,  test  assessments  may  be  biased,  and  systems  may 
be  produced  and  deployed  with  unknown  performance  limitations". 

c.  "Test  Agencies  are  not  in  compliance  with  10  U.S.C. 
section  2399,  Impartial  Contracted  Advisory  Assistance  and 
Services. 
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RECOMMENDATIONS  FOR  CORRECTIVE  ACTION 


1.  The  Director  of  Operational  Test  and  Evaluation  in  conjunction 
with  the  Deputy  Director,  Defense  Research  and  Engineering  (Test 
and  Evaluation) : 

a.  Require  program  management  offices  and  Operational  Test 
Agencies  to  identify  all  advisory  and  assistance  services 
contractors  and  subcontractors  that  participate  in  the  development, 
production,  or  testing  and  in  the  operational  test  and  evaluation 
for  major  defense  acquisition  systems  in  the  system* s  Test  and 
Evaluation  Master  Plan  (TEMP) * 

OPTBC  COMMENT: 

Nonconcur.  The  TEMP  is  a  planning  document  for  Test  and 
Evaluation.  The  staffing  and  approval  process  is  already  lengthy 
and  involved.  The  process  need  not  be  made  more  cumbersome  by 
causing  the  Services  to  rely  upon  the  TEMP  as  an  internal  control 
document  for  contract  support  activities. 

Requiring  program  offices  and  test  and  evaluation  activities  to 
identify  all  contractors  and  subcontractors  in  the  TEMP  offers  no 
assurance  against  the  involvement  of  a  contractor  in  the  development 
as  well  as  the  Operational  Test  and  Evaluation  of  a  particular 
system.  It  can  only  negatively  impact  the  currency,  relevancy  and 
utility  of  the  TEMP  as  a  planning  document  for  Test  and  Evaluation. 

Enforcing  compliance  with  public  law  governing  procurement 
operations  should  reside  with  those  officials  and  organizations 
responsible  for  their  administration.  This  can  be  accomplished 
through  the  application  of  appropriate  contract  clauses,  increased 
awareness,  oversight  and  internal  audits  as  required. 

AMC  COMMENT: 

Nonconcur.  The  TEMP  is  a  master  plan.  It  is  not  a  detailed 
test  planning  document.  The  TEMP  should  primarily  be  identifying 
the  overall  test  strategy  for  a  program  and  how  the  proposed  T&E 
will  support  the  acquisition  milestones.  A  detailed  list  of 
contractors  and  subcontractors  is  out  of  place  in  such  a  document. 
The  TEMP,  as  the  principal  T&E  planning  document,  is  required  to  be 
developed  prior  to  MS  I,  program  initiation,  prior  to  appointing  a 
PM  for  the  program.  At  the  time  the  TEMP  is  written,  there  are  no 
program  contracts  because  the  program  has  not  yet  started.  The 
TEMP  is  only  required  to  be  updated  at  program  milestones  or  when  a 
major  change  in  the  program  T&E  strategy  has  occurred.  For  that 
reason,  the  TEMP  is  not  the  most  suitable  place  to  maintain  a  record 
of  the  different  service  contracts  are  are  awarded  through  the  life 
of  a  program.  Lastly,  the  list  of  contractors  and  subcontractors 
can  be  enormous,  numbering  in  the  hundreds  when  the  entire 
developmental  effort  is  taken  into  account.  The  TEMP  preparer 
should  not  be  expected  to  determine  this  list  or  be  held  accountable 
for  its  completeness  or  accuracy. 
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b»  Propose  legislation  that  would  allow  the  Operational  Test 
Agencies  to  obtain  waivers  from  the  Director  of  Operational  Test 
and  Evaluation  to  use  the  same  advisory  and  assistance  service 
contractors  that  participated  in  the  development,  production  or 
testing  to  support  the  Operational  Test  and  Evaluation  under 
justifiable  conditions  when  sufficient  steps  have  been  taken  to 
ensure  the  impartiality  of  the  contractors  services. 

OPTEC  COMMENT: 

Concur  with  Comment.  Operational  Test  and  Evaluation  activities 
should  be  permitted  to  obtain  the  services  from  the  same  contractor, 
in  selected  instances  when  the  contractors  impartiality  can  be 
demonstrated.  Contractors,  by  virtue  of  their  earlier  involvement 
in  the  program,  the  highly  specialized  nature  of  the  services 
provided,  and  the  technical  knowledge  and  expertise  that  cannot  be 
otherwise  acquired  are  sometimes  imperative  to  the  efficient  and 
cost  effective  transfer  of  technical  knowledge  and  skills  associated 
with  a  weapon  system.  The  same  contractors  involvement  may  also  be 
desirable  when  cost  consideration  mandates  the  same  instrumentation 
be  used  for  both  technical  and  operational  testing. 

Approval  authority  should  reside  within  the  Service  requiring 
the  support,  with  periodic  reporting  requirement  to  the  director  of 
Operational  Test  and  Evaluation.  The  DOT&E  cannot  be  expected  to 
assess  the  merit  of  such  request,  and  should  not  be  accountable  for 
the  approval/disapproval  determination.  He  is  able  to  ensure  the 
arrangement  is  not  being  abused  by  monitoring  the  frequency  of  such 
request,  and  the  contractors  and  programs  involved. 

c.  Develop  a  standard  organizational  conflict  of  interest 
clause  that  precludes  advisory  and  assistance  services  contractors 
and  subcontractors  from  participating  in  development,  production,  or 
testing  and  Operational  Test  and  Evaluation  for  the  same  systems 
unless  a  waiver  is  obtained. 

OPTEC  COMMENTS: 

Concur  with  comment.  OPTEC  contracts  include  an  OCI  clause 
which  we  believe  is  effective.  A  second  standard  clause  requires 
the  prime  contractors  to  flow  down  the  OCX  clause  to  their 
subcontractors  intact.  We  believe  the  'clauses  are  working  as 
intended,  and  potential  conditions  for  OCI  are  being  reported. 
However,  this  does  not  preclude  efforts  on  the  part  of  DOD  to 
develop  a  more  effective  clause. 

2.  Commanders  of  the  Military  Departments’  Operational  Test 
Agencies: 

a.  Insert  organizational  conflict  of  interest  clauses  in 
existing  and  future  contracted  advisory  and  assistance  services 
contracts  that  preclude  contractors  and  subcontractors  from 
participating  in  development,  production,  or  test  and  Operational 
Test  and  Evaluation  of  the  same  system. 


OPTEC  COMMENT:  Concur.  OPTEC  contracts  currently  include  the 
clause  as  indicated.  Contracts  transferred  to  OPTEC  for 
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administration,  as  a  result  of  the  Test  and  Evaluation 
reorganization  will  be  amended  to  include  the  clause  consistent 
with  the  provisions  of  subpart  9.5  of  the  Federal  Acquisition 
Regulation. 

b.  Report  the  material  internal  control  weakness  of  using  the 
same  advisory  and  assistance  services  contractors  to  support 
Operational  Test  and  Evaluation  that  participated  in  development, 
production,  or  testing,  and  track  the  status  of  corrective  action 
taken  until  the  problems  noted  are  resolved. 

OPTEC  COMMENT:  This  command  does  not  concur  with  the  finding  that 
the  same  advisory  and  assistance  services  contractors  are  being  used 
to  support  the  two  different  activities  resulting  in  a  conflict  of 
interest. 
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Final  Report 
Page  No. 


Cost  Effectiveness  of  Services  Contracts 


1.  The  auditors  concluded  that  reliance  upon  services  contracts  to 
support  operational  test  and  evaluation  was  not  as  cost  effective  as 
developing  an  in-house  capability  to  perform  the  work.  They 
recommended  services  contracts  be  reduced  by  60  percent  and  an 
in-house  capability  be  developed.  OPTEC  endorses  the 
recommendation.  It  has  been  an  OPTEC  objective  for  some  time  to 
reduce  the  level  of  contractual  effort,  and  place  greater  reliance 
upon  in-house  resources.  However,  the  following  must  be  considered 
in  determining  the  appropriate  level  of  contract  support: 

a.  OPTEC  has  sustained  significant  reductions  in  personnel 
strength  over  the  past  year;  further  reductions  are  imminent. 
Concurrently,  significant  increases  in  mission  responsibility  have 
been  assigned. 

b.  A  reduction  of  60  percent  in  contractual  support  over  the 
next  five  years  is  a  reasonable  goal.  However,  it  is  impossible  to 
speculate  whether  this  reduction  will  generate  the  estimated 
savings.  Contractual  support  is  a  flexible  tool  for  the  tester  and 
evaluator,  and  to  some  extent  offers  a  considerable  economic 
advantage  to  the  government.  The  best  balance  between  in-house 
capability  and  the  level  of  contract  support  is  yet  to  be 
determined. 

2.  Page  41,  first  paragraph  implies  that  test  and  evaluation 
activities  are  contracting  out  functions  that  are  the  inherent 
responsibility  of  the  government.  DODD  4205.2  defines  basic 
Governmental  functions  as.,  "planning;  policy  development, 
interpretation,  and  enforcement?  program  and  budget  decision  making, 
and  finance  accountability. 

a.  OPTEC  is  not/has  not  entered  into  contractual  arrangements 
to  receive  these  or  similar  categories  of  services.  Neither  does 
OPTEC  permit  the  independent  evaluators  functions  to  be  performed 
by  contract.  These  functions  are  regarded  as  inherently  government 
responsibilities.  Published  policy  and  a  rigorous  review  process 
are  in  place  to  enforce  this  standard.  However,  OPTEC  does  contract 
in  many  areas  to  support  the  evaluators  efforts.  Examples  of 
conditions  justifying  contract  support  include: 

1.  Transfer  of  engineering  and  technical  knowledge 
pertinent  to  the  particular  program. 

2.  Special  knowledge  and  skills. 

3.  When  a  suitable  in-house  capability  does  not  exist. 

4.  Support  of  the  contractors  instrumentation  if  it  is 
the  sole  source  of  instrumentation. 
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b.  Under  no  circumstances  is  support  obtained  under  contract 
unless  it  is  economically  advantageous  to  the  government, 

(represents  a  cost  effective  solution)  the  capability  does  not  reside 
within  the  government,  and  task  completion  is  mission  essential) . 

c.  OPTEC,  in  addition  to  la(l)-(4)  above,  does  contract  for 
additional  personnel  to  accommodate  fluctuations  in  requirements, 
and  the  cyclic  nature  of  operational  test  and  evaluation.  This 
has  proven  to  be  a  cost  effective  solution?  it  is  not  to  circumvent 
personnel  ceilings  or  pay  limitations.  The  intensity  with  which 
OT&E  fluctuates,  in  terms  of  schedule,  categories  of  system  under¬ 
going  test,  system  technology  and  program  urgency  is  considerable 
and  program  slips  and  delays  are  commonplace.  It  is  not  feasible  to 
staff  for  all  contingencies;  neither  is  it  feasible  for  our 
contractor  to  do  so,  and  explains  their  reliance  on  subcontractors, 
particularly  in  high  specialized  areas  of  testing  technology. 

3.  The  report  concludes  OPTEC  is  obtaining  services  support  on  a 
repeated  or  extended  arrangement  (in  violation  of  4205.2). 

OPTEC  has  a  fixed  price  indefinite  delivery  contract.  Task 
orders  are  awarded,  under  that  contractor  on  a  time  and  materiel 
basis.  Each  task  order  has  a  fixed  price  and  a  fixed  period  of 
performance.  Each  task  order  clearly  describes  the  work  to  be 
performed  and  the  item  to  be  delivered.  Each  task  order  is 
carefully  screened  (internally)  to  ensure  compliance  with  DODD 
4205.2.  This  is  a  legitimate  application  of  contract  support.  It 
represents  an  efficient  and  cost  effective  application  of  DOD 
resources. 
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Cost  Effectiveness  of  Services  Contracts 
Recommendations  for  Corrective  Action 


CONCUR  COMPLETION 


ADDRESSEE 

RECOMMENDATION 

NONCONCUR 

DATE 

COMMENT 

DOT&E 

ASD(FM&P) 

OPTEC 

OPTECFOR 

AFOTEC 

Determine  in-house 
Civilian  requirements 
needed  to  perform 
mission 

concur 

DEC  91 

d:t&e 

AFD(FM&P) 

OPTEC 

OPTECFOR 

A  ■  OTEC 

Request  funding  to 
gradually  hire 
additional  civilian 
personnel . 

Concur 

FEB  92 

DOT&E 

ASD (FM&P) 
OPTEC 
OPTECFOR 
AFOTEC 

Reduce  the  use  of 

CAAS  contractors  by 

60%  over  next  5  years. 

Concur 

w/cmt 

DISCUSSION 
AT  ATTACH  1 

DOT&E 

ASD (FM&P) 

OPTEC 

OPTECFOR 

A  'OTEC 

Require  Military  Depart¬ 
ment  Operational  Test 
Agencies,  to  report  and 
control  their  advisory 
and  assistance  services 
contracts  to  support 
operational  test  and 
evaluation  as  contracted 
advisory  and  assistance 
services. 

Nonconcur 

See 

ATTACH  2  &  3 

Enclosure  4 
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COST  EFFECTIVENESS  OF  SERVICES  CONTRACTS 


1.  The  DODIG  report  concluded  that  operational  test  and 
evaluation  agencies  were  using  repeated  and  extended  service 
contracts  to  support  operational  tests  that  were  not  as  cost 
effective  as  developing  an  in-house-capability  to  perform  the 
same  work.  They  estimated  that  the  DOT&E  and  the  services  test 
agencies  could  save  8  million  annually  by  reducing  service 
contracts  by  €0  percent  and  developing  an  in-house  capability  to 
plan,  analyze  and  report  operational  test.  They  recommended  the 
DOT&E  and  the  Commanders  of  the  military  departments'  operational 
test  agencies  in  conjunction  with  ASD  (Force  Management  and 
Personnel) : 

a.  Determine  in-house  personnel  requirements  needed  to 
perform  their  mission. 

b.  Request  funding  to  gradually  hire  additional  civilian 
personnel. 

c.  Reduce  the  use  of  advisory  and  assistance  services 
contractors  by  60  percent  over  the  next  five  years. 

OPTEC  COMMENT: 

1.  This  command  strongly  endorses  the  DODIG* s  Recommendations. 

We  are  anxious  to  bring  more  work  in-house,  and  concurrently 
become  less  reliant  upon  contract  support.  However,  prior  to  full 
implementation  of  the  DODIG  recommendations,  the  following  must 
be  taken  into  consideration: 

a.  The  OPTEC  civilian  end  strength  has  been  reduced  by  278 

spaces  in  response  to  DMRD  936,  implemented  in  October  1990.  . 

Reductions  were  to  be  off-set  through  economies  and  efficiencies 
achieved  through  the  consolidation  of  organizations  and  mission 
responsibilities.  Further  reductions  are  planned  for  the  next 
fiscal  year.  Concurrently,  the  level  of  test  and  evaluation 
activity  has  remained  constant. 

b.  New  mission  responsibilities  are  currently  being 
assigned.  The  most  significant  of  these,  in  the  near  term,  is  the 
test  and  evaluation  of  all  IMA  programs. 
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2.  OPTEC  share  of  the  $8,000,000  program  annual  savings  is 
$1,560,317  according  to  annexes  provided  with  the  DODIG  report. 
This  allows  $5,630,083  for  personnel  salaries  to  offset  the  60 
percent  reduction  in  services  support,  or  the  addition  of  125 
personnel .spaces  to  the  TDA.  The  125  spaces  assumes  a 
distribution  of  grade  and  salary  level  comparable  to  that  of 
personnel  currently  assigned  (the  number  would  be  reduced  to 
approximately  80  if  the  command  were  to  focus  on  more  highly 
skilled  personnel).  $4,793,600  would  remain  for  contract 
support,  or  40  percent  of  the  current  level. 

a.  The  current  contractual  arrangement  permits  considerable 
flexibility  in  an  environment  that  mandates  the  same.  Services 
support  can  be  quickly  tailored  to  surge  for  requirements  over  a 
broad  range  of  skills,  to  include  low  density  skills,  and  over  an 
equally  broad  range  of  functional  areas.  The  requirement  may  be 
for  a  large  number  of  semi  skilled  data  collectors,  or  a  subject 
matter  engineering  expert  in  air  defense.  Additionally,  support 
arrangements  can  be  quickly  reconfigured  or  even  terminated  in 
response  to  the  dynamics  of  the  test  environment.  The  government 
pays  only  for  those  services  rendered,  whatever  the  circumstance. 

b.  Paragraph  2. a.  above  is  not  to  negate  the  advantages 
associated  with  the  DODIG  finding  and  recommendation.  The 
advantages  of  bringing  more  work  in-house  are  apparent,  and  it  is 
one  of  the  high  priority  objectives  of  this  organization. 
Paragraph  2. a.  above  is  intended  to  emphasize  that  speculating  on 
the  best  mix  of  contractor  support  and  in-house  capability  is 
more  difficult  that  the  DODIG  report  suggest. 
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DEPARTMENT  Of  THE  ARMY 
Office  Of  THt  UNOfft  ICCMftAftY 
WASHINGTON  OC  »8>lO  0101 

June  25/  1991 


SFUS-MIS 

MEMORANDUM  FOR  DIRECTOR,  TEST  AND  EVALUATION  MANAGEMENT  AGENCY 

SUBJECT:  Draft  Audit  Report  on  Consulting  Service  Contracts  for 
Operational  Teat  and  Evaluation  (Project  No.  OCH-5009) 

We  nonconcur  with  recommendation  2,  Part  B,  which  was  that 
the  auditor’s  "recommend  that  the  DoD  Director  of  Contracted 
Advisory  and  Assistance  Services/  in  conjunction  with  the 
Comptroller  of  the  Department  of  Defense,  require  the  Military 
Departments,  Operational  Test  Agencies  to  report  and  control 
their  advisory  and  assistance  services  contracts  to  support 
operational  test  and  evaluation  as  contracted  advisory  and 
assistance  services*"  According  to  the  Federal  Acquisition 
Regulation,  Section  37.204  Exclusions,  paragraph  (1)  "Those 
support  services  of  a  managerial  or  administrative  nature 
performed  as  a  simultaneous  part  of,  and  nonseparable  from 
specific  development,  production,  or  operational  support 
activities",  are  excluded  from  the  definition  of  advisory  and 
assistance  services.  Based  on  this  interpretation  of  the 
definition  and  the  information  provided  in  the  audit  we  do  not 
believe  that  the  services  identified  in  this  audit  should  be 
reported  or  controlled  as  Contracted  Advisory  and  Assistance  * 


Services  (CAAS). 


william  D.  Barr,  Chief, 

Study  Management  Branch,  Model 
Improvement  and  Study  Management  Agency 
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The  DODIG  report  recommends  that  the  DOD  Director  of  Contracted 
Advisory  and  Assistance  Services,  in  conjunction  with  the 
Comptroller  of  the  Department  of  Defense,  require  the  military 
departments  operational  test  agencies  to  report  and  control 
advisory  and  assistance  services  contracts  to  support  operational 
test  and  evaluation  as  contracted  advisory  and  assistance  services. 

OPTEC  COMMENT!  NONCONCUR. 

1.  The  Congress  has  specifically  identified  and  legitimized  a 
category  of  contract  activity,  that  otherwise  may  be  considered 
•’Personal  Services"  in  nature  and  therefore  in  violation  of  the 
Federal  Acquisition  Regulation  (FAR)  unless  specifically  authorized 
by  statute.  This  category  of  contracts,  titled:  Advisory  and 
Assistance  Services"  (CAAS)  is  defined  in  the  FARas  "services  to 
support  or  improve  agency  policy  development,  decision  making, 
management,  and  administration,  or  to  support  or  improve  the 
operation  of  management  systems". 

2.  It  is  not  the  intent  of  the  Congress  to  preclude  contractor 
involvement  in  support  of  the  materiel  acquisition  process,  knowing 
his  involvement  is  fundamental  to  that  same  process.  Rather  the 
intent  is  to  acknowledge  the  necessity  for  a  specific  category  of 
service  available  only  from  the  public  sector  while  maintaining  the 
integrity  of  federal  laws  and  statutes,  subject  to  a  set  of 
specific  conditions. 

3.  CAAS  provides  a  legitimate  necessary  means  to  improve 
government  services  and  operations  by  "obtaining  outside 
information,  points  of  view,  advice,  opinions,  recommendations  to 
enhance  understanding"  etc.  The  intent  of  the  Congress  is  to 
acknowledge  the  necessity  for  these  kinds  of  services  while 
enforcing  stringent  prohibitions  against  personal  services 
contracting  as  outlined  in  the  FAR  OPTEC  may  contract  for  these 
services  but  very  infrequently,  and  only  when  special  circumstances 
prevail . 

4 .  OPTEC  contracts  for  engineering  and  technical  services  provided 
by  engineering  service  contractors  to  support  operational  test  and 
evaluation  (TfcE) .  T&E  is  a  fundamental  imperative  to  final 
government  acceptance  of  a  complete  hardware  system;  it  is  mandated 
by  public  law,  and  a  function  of  the  research,  development, 
production  and  procurement  process.  Engineering  services  contracts 
in  support  of  materiel  acquisition  are  specifically  excluded  from 
this  category  of  CAAS  contracts  by  the  Federal  Acquisition 
Regulation  (FAR  37.203d). 
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5.  Contracts  for  technical  services  provided  after  government 
acceptance  of  a  complete  hardware  system  are  also  excluded  where 
they  are  procured  to  increase  the  original  design  performance 
capabilities  of  a  new  or  existing  system,  and  have  been  formally 
reviewed  and  approved  in  the  acquisition  planning  process.  It  is 
reasonable  to  conclude  from  the  above  discussion  that  OPTEC 
contracts  relied  upon  to  support  "Army  User  Testing"  as  defined  in 
Army  Regulations;  and  DOD  Publications  are  not  "Advisory, 
Assistance  and  Services"  contracts. 


no 
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DEPARTMENT  OF  THE  NAVY 

COMtUMOCH  OPVU-nONAI.  TUT  AND  (VALUATION  tOmCT 
NOKTOCN.  «IMmu  (Ml  l-MM 


Ser  02/  1  o  e  1 

01.  5  1991 


From:  Commander,  Operational  Test  and  Evaluation  Force 
To:  Inspector  General 

Subj:  DRAFT  AUDIT  REPORT  ON  CONSULTING  SERVICES  CONTRACTS  FOR 
OPERATIONAL  TEST  AND  EVALUATION  (PROJECT  NO.  0CH-5009) 

1.  Recommendations  la  -  lo  -  Do  not  concur .  Generally, 
C0M0PT8VF0R  program*  do  not  require  continual  analytical  support. 
In  a  program,  there  are  times  when  extensive  analytical  support 
is  required.  However  at  other  times r  no  analytical  involvement 
is  required.  Therefore  ,  contractors  era  only  used  when  there  is 
an  analytical  requirement  and  not  on  a  continual  basis* 
C0M0PTEVF0R  is  involved  with  numerous  programs  that  require 
various  types  of  specific  technical  expertise.  This  bakes  it 
Infeasible  to  ensure  that  there  are  sufficient  in-house  analysts 
with  a  specific  specialty  to  adequately  provide  the  necessary 
analytical  support.  Contractors  have  access  to  personnel  with 
various  backgrounds  and  levels  of  expertise.  Given  the  cyclical 
nature  of  the  analytical  requirements  coupled  with  the  varying 
technical  expertise  required,  the  discontinuation  of  contractor 
support  in  favor  of  maintaining  an  in-house  staff  of  analysts  is 
not  justified. 

2.  Recommendation  3a  -  Concur  in  principle.  COMOPTEVFOR 
presently  has  an  excellent  organizational  conflict  of  interest 
clause  that  is  a  part  of  sach  analytical  support  contract.  This 
clause  will  be  incorporated  into  the  revised  analytical  support 
instruction  to  be  completed  in  July  1991. 

3.  Recommendation  3b  and  3c  -  Concur.  COMOPTEVFOR* s  Contract 
specialist  will  have  a  conference  with  the  prospective  contractor 
to  formally  notify  the  contractor  of  the  provisions  for  impartial 
contracted  advisory  and  assistance  services  in  10  u.S.C.  2399, 
prior  to  the  evard  of  all  future  contracts. 

4.  Recommendation  3d  -  Concur.  COMOPTEVFOR  is  actively  taking 

action  to  remove  the  existing  conflict  of  interest  with  regards 
to  Webster  Engineering,  subcontractor  Veda,  Inc.  and  a  new 
contract  will  be  awarded  by  August  1992.  _ 


A.  MARSHALL 
Deputy  Commander 
end  Chief  of  Staff 


Copy  to: 

CNO  (OP-913) 
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DEPARTMENT  OF  THE  AIR  FORCE 

WASHINGTON  DC  20330-1000 


orrtcc  Of  tmc  a**»*tant  sccrctary 


2  8  JUN  1991 

MEMORANDUM  FOR  SAF/FMP 

SUBJECT:  DoD(IG)  Draft  Audit  Report  On  Consulting  Services  Contracts  Operational 
Test  and  Evaluation  (Project  No.  OCH-5009). 

We  have  reviewed  the  subject  audit  report  and  concur  with  most  of  the 
recommendations.  Specific  comments  from  HQ  AFOTEC,  the  Air  Force  Operational 
Test  Agency  (AF  OTA),  are  attached. 

GENERAL  COMMENTS 

While  interpretation  of  the  law  as  it  pertains  to  contractor  involvement  have  been 
difficult  to  obtain,  we  feel  the  audit  is  based  on  an  overly  strict  interpretation  of  the  use 
of  contractors  in  OT&E.  Section  2399  of  10  U.S.C.  does  exclude  individual  persons, 
however  contractors  are  only  excluded  from  being  "involved  (in  any  way)  in  the 
establishment  of  criteria  for  data  collection,  performance  assessment,  or  evaluation 
activities  for  the  OT&E".  HQ  AFOTEC  found  no  instances  where  these  exciuded 
activities  were  performed  by  the  AF  contractors  cited  in  the  report. 

We  oppose  action  to  implement  the  findings  on  cost-effectiveness  of  service 
contracts  until  a  more  thorough  analysis  can  be  conducted. 

An  absolutely  clear  interpretation  of  the  law  governing  the  use  of  contractors  is 
needed  We  also  wholeheartedly  concur  in  the  recommendation  to  establish  DOT&E 
waiver  authority  to  allow  the  use  of  contractors  by  the  service  OTAs  when  adequate 
safeguards  can  be  established. 

Point  of  contact  for  this  action  is  Major  Stephen  M  DeFrank  Jr,  who  can  be 
reached  at  DSN  227-1165. 


1  Attachment 
AFOTEC  Comments 


FREDERICK  J.  FOSTER,  U  Col,  USAF 
Deputy  Director 
Test  and  Evaluation 
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AFOTEC  MANAGEMENT  COMMENTS 

REPORT  OF  AUDIT  PROJECT  NO.  OCH-5009 

Following  art  management  comments  on  the  Draft  Audit  Report  Part  II  •  Findings  and 
Recommendations: 

a.  Services  Contractors  Supporting  Operational  Tests  and  Development: 

(1)  The  AMRAAM  System--The  discussions  pertaining  to  Veda,  Inc.  and  Webster 
Engineering  do  not  pertain  to  the  Air  Force,  since  these  contracts  only  supported  Navy 
activities.  The  Air  Force  did  use  the  Navy’s  contact  with  McLaughln  lor  some  test  support, 
but  according  to  our  dialogues  with  Point  Mugu,  the  contract  did  include  an  organizational 
conflict  of  interest  clause  by  reference.  We  do  not  believe  a  conflict  of  interest  occurred. 

(2)  The  MILSTAR  System--The  report  states  that  "The. ..conflict  of  Interest  clause  (for 
the  three  MILSTAR  contractors)  should  hev#  precluded  the  contractors  from  supporting 
(our  OTAE)....:  The  following  comments  support  our  position  that  there  waa/ls  no  conflict 
of  internet  and  we  provided  effective  management  to  eliminate  potential  conflicts  of  interest 

(a)  Milstar  is  a  multibiMlon  dollar  developmant  program  which  has  amployed  and 
continues  to  employ  a  large  number  of  contractors  and  subcontractors.  We  are  primarily 
limited  to  a  small  pool  of  contractors  who  have  negotiated  contracts  with  AFOTEC.  In  the 
case  of  Milstar,  chocwing  contractors  who  have  no  Involvement  in  the  Milstar  development 
program  and  who  have  the  expertise  to  assist  in  our  operational  evaluations  Is  not  practical 
without  revising  how  toe  DoD  lets  contracts.  This  has  traditionally  been  a  problem  with 
our  nuclear  survivabity  support  In  general.  We  were  aware  of  al  toe  situations  concerning 
Milstar  cited  in  toe  report  with  toe  exception  of  toe  BDM  case.  We  have  examined  end 
documented  these  situations  periodically  and  concluded  there  was  no  conflict  of  interest 
in  each  case. 

(b)  The  subject  report  stales  that  BDM  was  involved  In  both  a  Mflstar  developmental 
contract  and  an  AFOTEC  Milstar  contract  While  this  ie  true,  AFOTEC's  aubtaak  with  BDM 
was  completed  prior  to  toe  Initiation  of  BDM's  developmental  contract  The  report  states 
that  the  Rockwell  International  Corporation  subcontracted  with  BDM  In  October  1990  to 
perform  Radiation  Lot  Acceptance  Testing  on  specific  parts  of  the  Air  Force  Mister  terminal 
program.  The  report  also  states  that  AFOTEC  Issued  BDM  four  task  orders  from  1986 
end  1 989  to  support  the  Milstar  system.  In  fact,  the  AFOTEC  BDM  Milstar  subtask 
terminated  27  Dec  89,  9  months  prior  to  BDM’s  Involvement  in  the  Milstar  developmental 
program.  There  was  clearly  no  conflict  of  interest  In  this  case. 

(c)  The  subject  report  states  that  BAH  was  Involved  in  both  a  MHstar  developmental 
contract  and  an  AFOTEC  Milstar  contract.  While  this  is  true,  due  to  the  limitations  of  toe 
two  contracts,  and  the  additional  restrictions  we  put  on  BAH.  there  were  no  conceivable 
situations  where  BAH  would  evaluate  their  own  work.  The  report  states  that  the  Joint 
Terminal  Program  Office  (JTPO)  contracted  with  BAH  In  October  1990  to  provide  system 
engineering,  configuration  management,  Integrated  logistics,  test  and  evaluation, 
developmant  and  analysis  for  Milstar  segments,  and  Milstar  terminal  interoperability  test 
planning  and  support.  The  report  also  states  BAH  contracted  with  AFOTEC  In  1986  and 
1989  to  support  the  terminal  and  mission  control  element  (MCE)  nuclear  survivability 
portion  of  toe  assessment  of  toe  Milstar  satellite.  We  were  aware  of  the  situation  and  fait 
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that  BAWs  involvement  with  the  JTPO  would  not  influence  their  nucfear  assessment  of  the 
Mllstar  ground  segment.  The  BAH  AFOTEC  Mllstar  support  is  strictly  limited  to  examining 
the  nuclear  survivability  aspects  ol  the  Milstar  terminals  and  MCEs.  To  preclude  any 
potential  conflict  of  interest,  we  additionally  established  guidelines  in  1967  for  the  conduct 
of  the  two  BAH  offices  supporting  Mllstar. 

(d)  The  subject  report  states  that  SAIC  was  involved  in  both  a  Milstar  developmental 
contract  and  an  AFOTEC  Mllstar  contract.  While  this  is  true,  the  two  contracts  were 
directed  at  two  different  segments  of  Milstar.  The  report  states  that  Lockheed  sub¬ 
contracted  with  SAIC  in  January  1989  to  conduct  specific  EMP  Analyses  for  the  Mllstar 
MCE.  The  report  also  states  that  SAIC  contracted  with  AFOTEC  in  1969  to  support  the 
survivability  portion  of  the  assessment  ol  the  Mllstar  satellite.  We  were  aware  of  this 
situation  and  felt  that  SAIC’s  Involvement  wflh  the  ground  based  MCE  would  not  Influence 
their  evaluation  of  the  satsMte.  The  MCE  and  satellite  are  two  distinct  development  efforts. 
Wo  chose  a  separate  contractor,  BAH,  to  support  our  survtvabHtty  assessment  of  the  MCE. 

(3)  The  Radar  Systenv-The  report  states  there  is  a  conflict  of  interest  Involving  SAIC’s 
work  for  the  OTH-B  prime  contractor,  GE  and  their  performance  as  a  General  Support 
Contractor  on  an  on-going  subtask  for  the  IOT&E  of  OTH-B.  We  do  not  believe  there 
islwas  a  conflct  of  interest  for  two  reasons:  first,  the  SAIC  subcontract  for  GE  was  begun 
in  December  1966  and  completed  in  March  1987.  This  was  approximately  2.5  years  prior 
to  award  of  the  AFOTEC  GSC  to  SAIC  In  August  1989.  This  was  an  ongoing  subtask 
passed  from  BDM  to  SAIC  with  the  GSC.  Second,  the  work  performed  for  GE  was  specific 
to  the  West  Coast  OTH-B  system.  AFOTEC  is  not  scheduled  to  conduct  OT  on  this  radar. 

(4)  Test  and  Evaluation  Master  Plans-We  agree  that  TEMPs  should  be  reviewed  prior 
to  contracting  for  OT&E,  to  ensure  measures  of  performance,  test  event  or  scenario 
descriptions,  resource  requirements,  and  test  limitations  are  understood  by  contracting 
personnel.  However,  we  see  no  requirement  to  include  Identification  of  services  contractors 
in  TEMPs.  and  disagree  that  test  agencies  have  no  means  to  identify  those  contractors 
who  participated  in  system  development. 

(5)  Waivers  For  Use  of  Service  Contractors-This  section  of  the  audt  claims  10  U.S.C. 
2399  states  the  Director  of  OT&E  has  waiver  authority  on  the  limitation  of  the  use  of 
services  contractors.  10  U.S.C.  2399(e)(2)  states  the  Director  of  OT&E  has  such  authority, 
but  only  for  contracts  he  has  negotiated  for  advisory  and  assistance  services.  It  makes 
no  provision  for  waivers  for  contractors  participating  in  testing  for  the  military  departments. 
Furthermore,  the  audit  points  out  there  are  no  provisions  for  the  test  agencies  to  use  the 
same  services  contractors  under  similar  conditions.*  If  sufficient  steps  have  been  taken 
to  ensure  the  impartiality  of  the  contractor  in  providing  services,  which  subsection  (2) 
requires,  then  this  point  is  moot. 

(6)  Organizational  Conflict  of  Interest-The  thrust  of  this  section  of  the  audit  is  that  the 
lack  of  a  standardized  conflict  of  interest  clausa  among  the  service  test  agencies  did  not 
always  preclude  potential  conflict  of  interest  situations.  We  don’t  believe  the  audit  has 
established  facts  to  support  this  contention.  However,  we  have  no  objection  to  the  concept 
of  a  standardized  conflict  of  Interest  clause. 

(7)  Recommendation  1 : 
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(i)  Partially  Concur.  Wt  have  no  objection  to  identifying  all  contractors  and 
subcontractors  who  participate  in  OTAE  in  the  TEMP.  However,  identifying  contractors  who 
participated  in  development,  production,  or  development  testing  would  be  of  limited  utility 
to  the  OTAE  community  since  the  TEMP  is  updated  infrequently.  We  would  not  refer  to 
the  TEMP  for  this  information. 

(b)  Concur.  We  believe  legislation  to  clarify  existing  law  in  this  area  would  be 
extremely  beneficial,  and  we  support  the  proposal  for  legislation  as  phrased  in  the  audit 

(c)  Concur.  APOTEC  has  no  objection  to  a  standardized  organizational  conflict  of 
interest  clause. 

(8)  Recommendation  2— Concur  with  the  recommendation  as  written. 

(9)  Recommendation  3: 

(a)  Concur.  Instructions  will  be  provided  to  appropriate  contracting  activities 
regarding  the  specific  clause  to  be  used  in  each  contract. 

(b)  Concur.  Instructions  wil  be  provided  to  appropriate  contracting  actMttee  to  make 
this  notification. 

(c)  Concur.  AFOTEC  presently  takes  action  to  enforce  the  provisions  of  10  U.S.C. 
2399  and  all  other  pertinent  legislation  regarding  conflict  of  interest  through  contracting 
agencies,  and  will  continue  to  enforce  these  provisions. 

(d)  Concur  with  comments.  APOTEC  w*  report  material  internal  control  weaknesses 
in  those  cases  where  we  feel  adequate  safeguards  to  mitigate  conflict  of  interest  cannot 
be  assured.  However,  we  disagree  with  the  concept  of  such  reporting  for  al  circumstances 
where  contractors  supporting  OTAE  activities  had  involvement  with  development,  production, 
or  testing,  when  adequate  safeguards  are  in  place. 

b.  Cost-Effectiveness  of  Services  Contracts: 

(1)  General  Comments:  Several  reasons  exist  for  using  extended  services  contracts 
to  support  operational  testing  and  evaluation.  AFOTEC  support  contracts  are  technical- 
and  scientific-type  contracts  which  provide  our  functional  elements  specialized  expertise. 
Specialized  expertise  is  needed  to  structure  an  effective  OTAE  on  the  complex  weapon 
systems  being  acquired  to  counteract  the  sophisticated  threats  of  the  future.  General  work 
areas  include  concept  development,  test  planning,  test  execution,  data  management, 
modeling/simulation,  and  survivability  analysis.  Our  support  la  usually  provided  through 
long-term  (2-5  years)  tevei-ol-eftort  contracts  with  a  broad  scope,  general  work  areas,  and 
subtask  statement  provisions.  These  contracts  have  been  extremely  effective  because  of 
uncertainty  of  our  work  load  and  the  need  for  quick  reaction  support  when  unexpected 
requirements  arise.  On  certain  OTAE  programs,  we  have  Issued  contracts  to  accompflah 
specific  tasks  for  that  program.  Those  contracts  are  pursued  when  we  have  solid 
requirements  in  sufficient  time  to  support  the  procurement  lead  times.  Furthermore, 
AFOTEC  specifically  contracted  for  companies  with  differing  specialties  In  the  operational 
and  testing  area.  These  companies  were  asked  to  state  the  capabilities  In  terms  of 
meeting  surge  requirements.  AFOTEC  believes  we  can  only  get  limited  expertise  in- 
house  whether  civil  servant  or  military.  The  general  and  special  support  contracts  alow 
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for  the  use  of  entire  companies  to  respond  to  highly  technical  issues  on  an  as-needed 
basis.  If  we  don't  need  the  work  done,  we  don't  pay.  We  would  disagree  that  the 
expertise  experience  and  quality  is  available  through  the  military  or  civilian  personnel 
system.  Physicist,  scientist,  statisticians,  engineers,  etc.  of  this  caliber  are  often  paid  well 
for  their  work,  and  we  do  not  believe  that  we  could  attract  and  maintain  a  stable  force 
without  major  changes  in  the  pay  system. 

(2)  Background:  This  section  of  the  audit  synopsizes  pertinent  sections  of  DoD  Directive 
42052,  *DoD  Contracted  Advisory  and  Assistance  Services  (CAAS).’  There  are  basically 
four  categories  of  CAAS  services,  and  contractor  support  for  OT&E  is  not  covered  in 
categories  A.,  B.,  or  C.  Category  D.,  which  covers  engineering  and  technical  services 
contracts,  does  not  dearly  indude  contractor  support  to  operational  testing;  in  fact,  in  our 
view  contact  support  to  OT&E  is  not  covered,  in  our  opinion,  the  definition  of  CAAS  does 
not  apply  to  these  services  at  all. 

(3)  Repeated  and  Extended  Services  Contracts:  Repeated  and  extended  use  of  a 
services  contract  is  only  a  problem  if  the  contract  faKs  kilo  the  category  of  CAAS. 

Through  the  use  of  options 

on  general  and  special  support  contracts,  AFOTEC  guarantees  adequate  protection  to  the 
Government  for  termination  of  contract  services  when  warranted,  while  providing  minimal 
disruption  in  mission  support  through  continuity  of  the  contractor  work  force.  It  would  not 
be  mission  responsive  or  coat  effective  to  contract  in  a  'start  and  atop*  manner  tor  ongoing 
areas  of  technical  expertise  required  by  multiple  OT&E  programs. 

(4)  Services  Contracts  were  not  cost-effective.  We  strongly  disagree  with  this  statement 
The  contracts  used  by  AFOTEC  are  only  used  on  an  as-needed  basis.  We  only  pay  if 
we  need  support  that  is  covered  in  the  scope  of  either  the  general  or  special  support 
contracts.  If  congress,  DoD,  or  the  Air  Force  elects  to  cut  back  spending  In  any  area 
under  which  AFOTEC  has  a  contract  very  few  people  are  affected  compared  to  a  reaction 
in  force.  In  addition,  the  contractors  are  not  promised  anything  more  than  the  cost  of 
establishing  an  office  in  the  local  area.  If  our  funds  are  cut  by  millions  of  dollars,  then 
testing  will  cease,  and  programs  will  be  placed  on  hold  until  such  time  as  funding  is 
available.  This  arrangement,  considering  the  state  of  the  economy,  is  the  beat  situation 
we  co Lid  possible  be  In.  WHh  such  an  arrangement  lermlnalton  cools  are  minimal.  If  that 
becomes  necessary,  no  one  from  the  government  side  Is  laid  off,  and  virtually  all 
employment  risk  rests  with  the  contractor. 

(5)  Eimtoate  organizational  confltots  of  interest:  We  dtoagree  that  organizational  conflict 
of  interest  exists  today.  The  allegation  that  the  corporate  knowledge  needed  to  plan, 
analyze,  and  report  operational  tests  resides  with  the  contractor  work  force  is  only  partialy 
correct.  The  Air  Force  believes  that  an  appropriate  balance  between  contractor  and 
government  participation  on  OT&E  activities  exists  today.  The  allegation  that  the  potential 
for  organizational  conflict  of  Interest  can  be  eliminated,  which  this  section  of  the  audit 
maintains,  would  only  be  true  if  al  contractor  services  were  eliminated  tor  OT&E  in  favor 
of  In-house  capability.  This  extreme  is  dearly  unwarranted  and  not  cost  effective. 

(6)  Service  contracts  to  support  operational  tests  were  not  controlled  or  reported  as 
CAAS:  Because  we  do  not  agree  that  our  support  contracts  for  OT&E  qualify  as  CAAS, 
the  question  of  CAAS  reporting  is  moot  This  section  illustrates  the  difference  of  opinion 
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that  exists  between  the  Air  Force  and  the  DoDIG  regarding  the  proper  definition  of  CAAS. 
This  points  up  again  the  need  for  dear  interpretation  of  governing  legislation. 

(7)  Office  of  Management  and  Budget  Guidance:  No  comments. 

(8)  Recommendation  1: 

(a)  Concur.  AFOTEC  recommends  an  in  depth  analysis  of  the  costs  and  benefits 
associated  with  reduced  levels  of  contractor  dependence,  and  supports  this  recommendation 
in  that  context. 

(b)  Nonconcur.  This  recommendation  Is  premature  pending  the  completion  of  the 
analysis  cited  In  paragraph  (a)  above. 

(c)  Nonconcur.  AFOTEC  disagrees  with  actions  to  adjust  contracting  levels  or 
funding  pending  the  results  of  the  in  depth  analysis.  AFOTEC  nonconcurs  with  the 
estimated  monitory  benefits  cited  In  this  section  and  Illustrated  In  Appendix  Q  of  the  audit, 
based  on  our  belief  the  audit  analysis  of  these  savings  is  incomplete. 

(9)  Recommendation  2:  Concur  with  comments.  AFOTEC  supports  efforts  to  clarify 
the  definition  and  scope  of  CAAS  contracts.  If  the  decision  is  made  that  general  and 
special  support  contracts  supporting  OT4E  indeed  fall  under  the  definition  of  CAAS,  then 
we  would  concur  with  this  recommendation. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON,  DC  20301 


11  July  1991 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE 

SUBJECT:  Draft  Audit  Report  on  Consulting  Services  Contracts  for 
Operational  Test  and  Evaluation  (Project  No.  0CH-5009) 

This  memorandum  responds  to  your  request  for  comments  on  subject 
draft  report. 

Pfrv-rwuandition  B.2.  The  audit  recommends  that  the  Director,  DoD 
Contracted  Advisory  and  Assistance  Services,  and  the  Compt roller, 

DoD,  require  the  Military  Departments'  Operational  Test  Agencies  to 
report  and  control  their  advisory  and  assistance  services  contracts 
to  support  operational  test  and  evaluation  as  contracted  advisory  and 
assistance  services  (CAAS) . 

Nonconcur ,  We  cannot  require  that  the  test  agencies  report  those 
advisory  and  assistance  services  efforts  that  are  currently 
exempted/excluded  from  the  definition  of  CAAS.  The  current 
definition  provides  specific  exclusions  that  can  be  reasonably 
applied  to  contractor  support  used  by  the  test  agencies.  For 
example,  enclosure  (3),  paragraph  A,  DoD  Directive  4025.2  "DoD 
Contracted  Advisory  and  Assistance  Services",  excludes  from  the  scope 
of  CAAS  studies,  analyses  and  evaluation,  "system  specific 
engineering  studies".  Subpart  37.204  of  the  Federal  Acquisition 
Regulation  excludes  "engineering  studies  related  to  specific  physical 
or  performance  characteristics  of  existing  or  proposed  systems." 
However,  there  are  ongoing  initiatives  to  strengthen  the  management, 
identification  and  reporting  of  CAAS.  A  major  task  is  to  develop  an 
easier  to  apply  CAAS  definition  and  ensure  that  it  is  consistently 
applied  throughout  DoD.  We  plan  to  publish  and  have  in  place  the 
improved  definition  in  the  revised  DoD  Directive  4205.2,  by  October 
1,  1991.  We  believe  this  will  help  to  resolve  the  uncertainty  of 
what  is  or  not  CAAS. 


Robert  A.  Nemetz 

Director,  DoD  Contracted  Advisory 
and  Assistance  Services 
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